CN114842078A - Dual-channel satellite attitude estimation network based on deep learning - Google Patents
Dual-channel satellite attitude estimation network based on deep learning Download PDFInfo
- Publication number
- CN114842078A CN114842078A CN202210393253.4A CN202210393253A CN114842078A CN 114842078 A CN114842078 A CN 114842078A CN 202210393253 A CN202210393253 A CN 202210393253A CN 114842078 A CN114842078 A CN 114842078A
- Authority
- CN
- China
- Prior art keywords
- satellite
- information
- model
- attitude estimation
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013135 deep learning Methods 0.000 title claims abstract description 13
- 238000012549 training Methods 0.000 claims abstract description 18
- 238000012360 testing method Methods 0.000 claims abstract description 16
- 238000007781 pre-processing Methods 0.000 claims abstract description 9
- 238000012545 processing Methods 0.000 claims abstract description 7
- 238000012795 verification Methods 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 6
- 239000000284 extract Substances 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000004088 simulation Methods 0.000 claims description 3
- 238000000034 method Methods 0.000 abstract description 12
- 238000001514 detection method Methods 0.000 abstract description 4
- 238000005070 sampling Methods 0.000 abstract 2
- 230000006870 function Effects 0.000 description 10
- 238000004519 manufacturing process Methods 0.000 description 4
- 229940054870 urso Drugs 0.000 description 2
- RUDATBOHQWOJDD-UZVSRGJWSA-N ursodeoxycholic acid Chemical compound C([C@H]1C[C@@H]2O)[C@H](O)CC[C@]1(C)[C@@H]1[C@@H]2[C@@H]2CC[C@H]([C@@H](CCC(O)=O)C)[C@@]2(C)CC1 RUDATBOHQWOJDD-UZVSRGJWSA-N 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000010219 correlation analysis Methods 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Abstract
A two-channel satellite attitude estimation network based on deep learning is disclosed. The network flow comprises the following steps: the method comprises the steps of constructing a satellite attitude estimation data set, dividing, preprocessing data, extracting image features by a ResNet model, learning spatial position information by using an improved Vision transform model, learning rotation information by using a Hourglass network, respectively outputting position information and attitude information of a satellite, calculating the distance between an output value and a label, reversely transmitting the distance to the model for iterative training to obtain an optimal model, and evaluating the model by using a test set. The core module self-attention mechanism of the improved Vision Transformer model can focus on the position of a satellite in an image, and the Hourglass network learning repeated up-sampling and down-sampling processing means can effectively infer key points of the satellite contour. The invention uses the two-channel network to learn the space position information and the attitude information of the satellite respectively, thereby effectively avoiding the mutual influence between the two information, improving the estimation accuracy of the satellite attitude and providing a new intelligent means for the space non-cooperative target detection.
Description
Technical Field
The invention relates to the field of target detection, in particular to a two-channel satellite attitude estimation network based on deep learning.
Background
The satellite attitude estimation based on vision is a difficult problem to be solved urgently in the field of aerospace, and has important application values in the aspects of navigation, on-orbit maintenance, space rubbish cleaning and the like. However, the satellite attitude estimation based on pure vision faces a plurality of technical problems to be solved, such as inconvenience brought to camera imaging by a space illumination environment, mutual influence of spatial position information and self-rotation information of a satellite, and the like, and the problems bring a plurality of challenges to a satellite attitude estimation solution based on pure vision.
The Vision Transformer simulates the attention mechanism of the human brain, can automatically focus on the detected object when learning the image characteristics, and can effectively identify the spatial position information of the satellite body in the shot object. The Hourglass is widely used in the field of image key point detection, and the method has the characteristics that key point information of the satellite contour can be effectively detected, the rotation angle of the satellite in the image relative to a reference coordinate can be calculated through learning the key points of the satellite contour, and then the rotation information of the satellite can be learned. The two-channel network is used for effectively decoupling the spatial position information and the self rotation information of the satellite, and mutual interference between the two kinds of information is reduced.
The invention patent of the granted publication No. CN 109827578B discloses a "satellite relative attitude estimation method based on contour similarity", which performs similarity analysis on satellite image contours by using simulated images for projection, but the method is deficient in recognition efficiency and robustness. The invention patent of the grant publication number CN 105300384B discloses an interactive filtering method for satellite attitude determination, which determines satellite attitude information by collecting relevant data of a satellite sensor, and the method is only used for measuring the self attitude of a satellite body and cannot identify non-cooperative satellite attitude information.
Disclosure of Invention
Objects of the invention
The invention aims to provide a two-channel satellite attitude estimation network based on deep learning. The invention provides a double-channel deep learning network by shooting a non-cooperative satellite image through a camera, firstly extracting image characteristics by using a ResNet model, and then learning position information and rotation information of a satellite by using the double-channel network, wherein an improved Vision transform model is used for learning satellite spatial position information, a Hourglass network is used for learning satellite attitude information, the spatial position information and self rotation information of the satellite are effectively decoupled, and a new solution is provided for satellite attitude estimation and even non-cooperative target detection.
(II) technical scheme
The technical scheme of the invention is that a two-channel satellite attitude estimation network based on deep learning is characterized by comprising the following steps: the method comprises the steps of manufacturing a satellite attitude data set, dividing the data set into a training set, a verification set and a test set, preprocessing image information, inputting the preprocessed training set and the preprocessed verification set into a ResNet model to extract image features, learning satellite spatial position information by using an improved Vision Transformer model, learning satellite attitude information by using a Hourglass network, calculating the distance between a model predicted value and a label, reversely returning the model to the network to perform iterative training to obtain an optimal satellite attitude estimation model, and finally inputting the test set into the optimal model to evaluate the performance of the model.
The method comprises the steps of constructing a satellite attitude rotation data set, preprocessing satellite attitude image data, extracting local features of a satellite image through a convolutional neural network, carrying out position coding on the local features through a position coder, carrying out feature extraction again on the coded image features through a nonlinear residual error self-attention mechanism, processing the features through a post-processing module, and finally outputting attitude prediction information through a full connection layer.
And (3) making a satellite attitude estimation image data set, adopting a simulation image, marking the attitude information of the satellite in the image and making a label.
And dividing the satellite attitude estimation image data set into a training set, a verification set and a test set according to a certain proportion.
And (3) carrying out data preprocessing on the satellite attitude estimation image data set, and standardizing the image and the label so that the image and the label can be input into network training.
And inputting the training set and the verification set subjected to data preprocessing into a ResNet model, extracting the satellite image features by using components such as a convolution layer, a pooling layer and residual connection of the ResNet model, and using residual networks such as ResNet-18, ResNet-34 and ResNet-50.
The improved Vision Transformer model is used for learning satellite spatial position information, and simultaneously a Hourglass network is used for learning satellite attitude information, wherein compared with the traditional Vision Transformer, the improved Vision Transformer firstly compresses and extracts a four-dimensional matrix into a three-dimensional matrix after processing ResNet model characteristics, namely [ batch, channel, length and width ] is compressed and extracted into [ batch, length and channel width ], so that the problem that the traditional Vision Transformer compresses the image length and width into a column to input the characteristics into a self-power mechanism to destroy the spatial position of an object in the image can be effectively avoided.
The network respectively outputs the spatial position information and the attitude information of the satellite, calculates the distance between the output information and the marked attitude information, reversely transmits the distance information to the network, and performs optimization iterative training to obtain an optimal satellite attitude estimation model.
And inputting the preprocessed test set into a satellite attitude estimation model to obtain attitude estimation information of the test set, and comparing the attitude estimation information with the attitude information marked by the test set, thereby evaluating the satellite attitude estimation model.
The invention realizes a two-channel satellite attitude estimation network based on deep learning, and provides a two-channel deep learning network.
(III) major advantages of the invention
The technical scheme of the invention has the following advantages: according to the method, correlation analysis is carried out aiming at the characteristics of satellite attitude information, the spatial position information and the rotation information of the satellite are successfully decoupled by using a two-channel network, the spatial position information of the satellite is learned by using an improved Vision Transformer model, the satellite attitude information is learned by using a Hourglass network, the design of the two-channel network and the advantages of each channel network accord with the characteristics of the satellite attitude information, and the satellite attitude estimation precision is effectively provided.
Drawings
FIG. 1 is a flow diagram of the present invention;
FIG. 2 is a partial sample graph of a URSO satellite attitude estimation data set in accordance with an embodiment of the present invention;
Detailed Description
In order to make the technical scheme, advantages and purposes of the invention clearer, the technical scheme of the invention is further explained by combining a method flow described by a specific example and referring to the attached drawings.
The embodiment 1 of the invention relates to a two-channel satellite attitude estimation network based on deep learning, which is shown in figure 1 and comprises the following steps:
the method comprises the steps of manufacturing a satellite attitude estimation image data set, adopting a simulation image, marking attitude information of a satellite in the image and manufacturing the satellite attitude estimation image data set into a label, and rendering a satellite picture into a vivid space environment by using non Engine 4 software when manufacturing the data set, or using a URSO satellite attitude estimation data set manufactured by university of California scholars by using non Engine 4 software.
The method comprises the steps of dividing a satellite attitude estimation image data set into a training set, a verification set and a test set according to the proportion of 7:2:1, wherein the training set is mainly used for training a satellite attitude estimation model, the verification set is input into the model along with the training set and is mainly used for adjusting model hyper-parameters, and the test set is mainly used for evaluating the performance of the trained satellite attitude estimation model.
And (3) carrying out data preprocessing on the satellite attitude estimation image data set, firstly processing the satellite image by using an image preprocessing technology, and processing the image and the label into a data type capable of being input into a deep learning network.
Inputting a training set and a verification set subjected to data preprocessing into a ResNet model, extracting satellite image features by using components such as a convolution layer, a pooling layer and residual connection of the ResNet model, specifically extracting the satellite image features by using ResNet-18, and removing a global average pool layer and a last complete connection layer of an original network to keep the spatial feature resolution.
The improved Vision Transformer model is used for learning satellite spatial position information, and simultaneously a Hourglass network is used for learning satellite attitude information, wherein compared with the traditional Vision Transformer, the improved Vision Transformer firstly compresses and extracts a four-dimensional matrix into a three-dimensional matrix after processing ResNet model characteristics, namely [ batch, channel, length and width ] is compressed and extracted into [ batch, length and channel width ], so that the problem that the traditional Vision Transformer compresses the image length and width into a column to input the characteristics into a self-power mechanism to destroy the spatial position of an object in the image can be effectively avoided.
The network outputs the spatial position information and the rotation information of the satellite respectively, and three loss functions are required to be designed, wherein the three loss functions are as follows: calculating a space position information loss function of a satellite; calculating a loss function of the rotation information of the satellite body; thirdly, calculating an overall loss function, wherein the function expressions are respectively as follows:
L＝αL pos +βL ori (α+β＝1) (3)
l, L therein pos 、L ori Respectively representing a total loss function, a space position information loss function of the satellite and a satellite body rotation information loss function,
and
respectively a spatial position tag and a rotational tag,
and
is the spatial information and rotational information output by the model.
And reversely transmitting the three loss functions to the network, performing optimization iterative training by using a gradient descent method, and performing iterative training for multiple times to obtain an optimal satellite attitude estimation model.
Inputting the preprocessed test set into the satellite attitude estimation model to obtain attitude estimation information of the test set, and comparing the attitude estimation information with attitude information marked by the test set, thereby evaluating the satellite attitude estimation model.
Those skilled in the art will appreciate that the invention may be practiced without these specific details.
Claims (2)
1. A two-channel satellite attitude estimation network based on deep learning is characterized by mainly comprising the following steps:
step 1, constructing a satellite attitude estimation image data set, adopting a simulation image, and marking attitude information of a satellite in the image;
step 2, dividing the data set into a training set, a verification set and a test set according to a certain proportion;
step 3, satellite data preprocessing;
step 4, inputting the processed training set and the processed verification set into a ResNet model, and extracting image characteristics by using components of a convolution layer, a pooling layer, residual connection and the like of the ResNet model;
step 5, learning satellite spatial position information by using an improved Vision Transformer model, and learning satellite rotation information by using a Hourglass network;
step 6, the network respectively outputs the space position information and the attitude information of the satellite, calculates the distance between the output information and the marked attitude information, reversely transmits the distance information to the network, and performs optimization iterative training to obtain an optimal satellite attitude estimation model;
and 7, inputting the preprocessed test set into the satellite attitude estimation model to obtain attitude estimation information of the test set, and comparing the attitude estimation information with the attitude information marked by the test set, thereby evaluating the satellite attitude estimation model.
2. The deep learning based two-channel satellite attitude estimation network of claim 1, wherein:
in step 3, the improved Vision transform model is used to learn the satellite spatial position information, wherein the improved Vision transform is compared with the traditional Vision transform, after processing the characteristics of the ResNet model, the improved Vision transform firstly compresses and extracts the four-dimensional matrix into a three-dimensional matrix, namely [ batch, channel, length, width ] into [ batch, length, channel width ] so as to effectively avoid the problem that the traditional Vision transform compresses the image length and width into a column to input the characteristics into the self-attention mechanism, thereby destroying the spatial position of the object in the image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210393253.4A CN114842078A (en) | 2022-04-14 | 2022-04-14 | Dual-channel satellite attitude estimation network based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210393253.4A CN114842078A (en) | 2022-04-14 | 2022-04-14 | Dual-channel satellite attitude estimation network based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114842078A true CN114842078A (en) | 2022-08-02 |
Family
ID=82565680
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210393253.4A Pending CN114842078A (en) | 2022-04-14 | 2022-04-14 | Dual-channel satellite attitude estimation network based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114842078A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116051632A (en) * | 2022-12-06 | 2023-05-02 | 中国人民解放军战略支援部队航天工程大学 | Six-degree-of-freedom attitude estimation algorithm for double-channel transformer satellite |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110503689A (en) * | 2019-08-30 | 2019-11-26 | 清华大学 | Attitude prediction method, model training method and device |
| CN110969124A (en) * | 2019-12-02 | 2020-04-07 | 重庆邮电大学 | Two-dimensional human body posture estimation method and system based on lightweight multi-branch network |
| CN112183637A (en) * | 2020-09-29 | 2021-01-05 | 中科方寸知微(南京)科技有限公司 | Single-light-source scene illumination re-rendering method and system based on neural network |
| CN113095106A (en) * | 2019-12-23 | 2021-07-09 | 华为数字技术（苏州）有限公司 | Human body posture estimation method and device |
| CN113850151A (en) * | 2021-09-02 | 2021-12-28 | 中山大学 | Method, device, terminal and storage medium for identifying distraction behavior of driver |
| CN114332639A (en) * | 2021-11-30 | 2022-04-12 | 中国人民解放军战略支援部队航天工程大学 | Satellite attitude vision measurement algorithm of nonlinear residual error self-attention mechanism |
-
2022
- 2022-04-14 CN CN202210393253.4A patent/CN114842078A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110503689A (en) * | 2019-08-30 | 2019-11-26 | 清华大学 | Attitude prediction method, model training method and device |
| CN110969124A (en) * | 2019-12-02 | 2020-04-07 | 重庆邮电大学 | Two-dimensional human body posture estimation method and system based on lightweight multi-branch network |
| CN113095106A (en) * | 2019-12-23 | 2021-07-09 | 华为数字技术（苏州）有限公司 | Human body posture estimation method and device |
| CN112183637A (en) * | 2020-09-29 | 2021-01-05 | 中科方寸知微(南京)科技有限公司 | Single-light-source scene illumination re-rendering method and system based on neural network |
| CN113850151A (en) * | 2021-09-02 | 2021-12-28 | 中山大学 | Method, device, terminal and storage medium for identifying distraction behavior of driver |
| CN114332639A (en) * | 2021-11-30 | 2022-04-12 | 中国人民解放军战略支援部队航天工程大学 | Satellite attitude vision measurement algorithm of nonlinear residual error self-attention mechanism |
Non-Patent Citations (1)
| Title |
|---|
| 李孟锡等: "基于HOURGLASS网络语义关键点提取的光学图像空间目标姿态估计方法", 《信号处理》, vol. 37, no. 9, pages 1653 - 1662 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116051632A (en) * | 2022-12-06 | 2023-05-02 | 中国人民解放军战略支援部队航天工程大学 | Six-degree-of-freedom attitude estimation algorithm for double-channel transformer satellite |
| CN116051632B (en) * | 2022-12-06 | 2023-12-05 | 中国人民解放军战略支援部队航天工程大学 | Six-degree-of-freedom attitude estimation algorithm for double-channel transformer satellite |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108537742B (en) | Remote sensing image panchromatic sharpening method based on generation countermeasure network | |
| CN110222591B (en) | Lane line detection method based on deep neural network | |
| CN111160297A (en) | Pedestrian re-identification method and device based on residual attention mechanism space-time combined model | |
| CN110599537A (en) | Mask R-CNN-based unmanned aerial vehicle image building area calculation method and system | |
| CN113780149B (en) | Remote sensing image building target efficient extraction method based on attention mechanism | |
| CN110675423A (en) | Unmanned aerial vehicle tracking method based on twin neural network and attention model | |
| CN111008583B (en) | Pedestrian and rider posture estimation method assisted by limb characteristics | |
| CN111681178B (en) | Knowledge distillation-based image defogging method | |
| CN114565655B (en) | Depth estimation method and device based on pyramid segmentation attention | |
| CN112365434B (en) | Unmanned aerial vehicle narrow passage detection method based on double-mask image segmentation | |
| CN114332639B (en) | Satellite attitude vision measurement method of nonlinear residual error self-attention mechanism | |
| CN114049356B (en) | Method, device and system for detecting structure apparent crack | |
| CN113112583B (en) | 3D human body reconstruction method based on infrared thermal imaging | |
| CN115131313A (en) | Hyperspectral image change detection method and device based on Transformer | |
| CN114842078A (en) | Dual-channel satellite attitude estimation network based on deep learning | |
| CN113743417A (en) | Semantic segmentation method and semantic segmentation device | |
| CN116524189A (en) | High-resolution remote sensing image semantic segmentation method based on coding and decoding indexing edge characterization | |
| CN115546223A (en) | Method and system for detecting loss of fastening bolt of equipment under train | |
| CN113516693B (en) | Rapid and universal image registration method | |
| CN114049541A (en) | Visual scene recognition method based on structural information characteristic decoupling and knowledge migration | |
| CN111223113B (en) | Nuclear magnetic resonance hippocampus segmentation algorithm based on dual dense context-aware network | |
| CN117132919A (en) | Multi-scale high-dimensional feature analysis unsupervised learning video anomaly detection method | |
| CN111709269A (en) | Human hand segmentation method and device based on two-dimensional joint information in depth image | |
| CN114898464B (en) | Lightweight accurate finger language intelligent algorithm identification method based on machine vision | |
| CN110210321B (en) | Under-sample face recognition method based on multi-dimensional scale transformation network and block weighting method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |