CN114842078A - Dual-channel satellite attitude estimation network based on deep learning - Google Patents

Dual-channel satellite attitude estimation network based on deep learning Download PDF

Info

Publication number
CN114842078A
CN114842078A CN202210393253.4A CN202210393253A CN114842078A CN 114842078 A CN114842078 A CN 114842078A CN 202210393253 A CN202210393253 A CN 202210393253A CN 114842078 A CN114842078 A CN 114842078A
Authority
CN
China
Prior art keywords
satellite
information
model
attitude estimation
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210393253.4A
Other languages
Chinese (zh)
Inventor
任元
叶瑞达
王煜晶
王卫杰
宋铁岭
王元钦
刘通
刘政良
刘钰菲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peoples Liberation Army Strategic Support Force Aerospace Engineering University
Original Assignee
Peoples Liberation Army Strategic Support Force Aerospace Engineering University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peoples Liberation Army Strategic Support Force Aerospace Engineering University filed Critical Peoples Liberation Army Strategic Support Force Aerospace Engineering University
Priority to CN202210393253.4A priority Critical patent/CN114842078A/en
Publication of CN114842078A publication Critical patent/CN114842078A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • G06T7/73Determining position or orientation of objects or cameras using feature-based methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/084Backpropagation, e.g. using gradient descent
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

A two-channel satellite attitude estimation network based on deep learning is disclosed. The network flow comprises the following steps: the method comprises the steps of constructing a satellite attitude estimation data set, dividing, preprocessing data, extracting image features by a ResNet model, learning spatial position information by using an improved Vision transform model, learning rotation information by using a Hourglass network, respectively outputting position information and attitude information of a satellite, calculating the distance between an output value and a label, reversely transmitting the distance to the model for iterative training to obtain an optimal model, and evaluating the model by using a test set. The core module self-attention mechanism of the improved Vision Transformer model can focus on the position of a satellite in an image, and the Hourglass network learning repeated up-sampling and down-sampling processing means can effectively infer key points of the satellite contour. The invention uses the two-channel network to learn the space position information and the attitude information of the satellite respectively, thereby effectively avoiding the mutual influence between the two information, improving the estimation accuracy of the satellite attitude and providing a new intelligent means for the space non-cooperative target detection.

Description

Dual-channel satellite attitude estimation network based on deep learning
Technical Field
The invention relates to the field of target detection, in particular to a two-channel satellite attitude estimation network based on deep learning.
Background
The satellite attitude estimation based on vision is a difficult problem to be solved urgently in the field of aerospace, and has important application values in the aspects of navigation, on-orbit maintenance, space rubbish cleaning and the like. However, the satellite attitude estimation based on pure vision faces a plurality of technical problems to be solved, such as inconvenience brought to camera imaging by a space illumination environment, mutual influence of spatial position information and self-rotation information of a satellite, and the like, and the problems bring a plurality of challenges to a satellite attitude estimation solution based on pure vision.
The Vision Transformer simulates the attention mechanism of the human brain, can automatically focus on the detected object when learning the image characteristics, and can effectively identify the spatial position information of the satellite body in the shot object. The Hourglass is widely used in the field of image key point detection, and the method has the characteristics that key point information of the satellite contour can be effectively detected, the rotation angle of the satellite in the image relative to a reference coordinate can be calculated through learning the key points of the satellite contour, and then the rotation information of the satellite can be learned. The two-channel network is used for effectively decoupling the spatial position information and the self rotation information of the satellite, and mutual interference between the two kinds of information is reduced.
The invention patent of the granted publication No. CN 109827578B discloses a "satellite relative attitude estimation method based on contour similarity", which performs similarity analysis on satellite image contours by using simulated images for projection, but the method is deficient in recognition efficiency and robustness. The invention patent of the grant publication number CN 105300384B discloses an interactive filtering method for satellite attitude determination, which determines satellite attitude information by collecting relevant data of a satellite sensor, and the method is only used for measuring the self attitude of a satellite body and cannot identify non-cooperative satellite attitude information.
Disclosure of Invention
Objects of the invention
The invention aims to provide a two-channel satellite attitude estimation network based on deep learning. The invention provides a double-channel deep learning network by shooting a non-cooperative satellite image through a camera, firstly extracting image characteristics by using a ResNet model, and then learning position information and rotation information of a satellite by using the double-channel network, wherein an improved Vision transform model is used for learning satellite spatial position information, a Hourglass network is used for learning satellite attitude information, the spatial position information and self rotation information of the satellite are effectively decoupled, and a new solution is provided for satellite attitude estimation and even non-cooperative target detection.
(II) technical scheme
The technical scheme of the invention is that a two-channel satellite attitude estimation network based on deep learning is characterized by comprising the following steps: the method comprises the steps of manufacturing a satellite attitude data set, dividing the data set into a training set, a verification set and a test set, preprocessing image information, inputting the preprocessed training set and the preprocessed verification set into a ResNet model to extract image features, learning satellite spatial position information by using an improved Vision Transformer model, learning satellite attitude information by using a Hourglass network, calculating the distance between a model predicted value and a label, reversely returning the model to the network to perform iterative training to obtain an optimal satellite attitude estimation model, and finally inputting the test set into the optimal model to evaluate the performance of the model.
The method comprises the steps of constructing a satellite attitude rotation data set, preprocessing satellite attitude image data, extracting local features of a satellite image through a convolutional neural network, carrying out position coding on the local features through a position coder, carrying out feature extraction again on the coded image features through a nonlinear residual error self-attention mechanism, processing the features through a post-processing module, and finally outputting attitude prediction information through a full connection layer.
And (3) making a satellite attitude estimation image data set, adopting a simulation image, marking the attitude information of the satellite in the image and making a label.
And dividing the satellite attitude estimation image data set into a training set, a verification set and a test set according to a certain proportion.
And (3) carrying out data preprocessing on the satellite attitude estimation image data set, and standardizing the image and the label so that the image and the label can be input into network training.
And inputting the training set and the verification set subjected to data preprocessing into a ResNet model, extracting the satellite image features by using components such as a convolution layer, a pooling layer and residual connection of the ResNet model, and using residual networks such as ResNet-18, ResNet-34 and ResNet-50.
The improved Vision Transformer model is used for learning satellite spatial position information, and simultaneously a Hourglass network is used for learning satellite attitude information, wherein compared with the traditional Vision Transformer, the improved Vision Transformer firstly compresses and extracts a four-dimensional matrix into a three-dimensional matrix after processing ResNet model characteristics, namely [ batch, channel, length and width ] is compressed and extracted into [ batch, length and channel width ], so that the problem that the traditional Vision Transformer compresses the image length and width into a column to input the characteristics into a self-power mechanism to destroy the spatial position of an object in the image can be effectively avoided.
The network respectively outputs the spatial position information and the attitude information of the satellite, calculates the distance between the output information and the marked attitude information, reversely transmits the distance information to the network, and performs optimization iterative training to obtain an optimal satellite attitude estimation model.
And inputting the preprocessed test set into a satellite attitude estimation model to obtain attitude estimation information of the test set, and comparing the attitude estimation information with the attitude information marked by the test set, thereby evaluating the satellite attitude estimation model.
The invention realizes a two-channel satellite attitude estimation network based on deep learning, and provides a two-channel deep learning network.
(III) major advantages of the invention
The technical scheme of the invention has the following advantages: according to the method, correlation analysis is carried out aiming at the characteristics of satellite attitude information, the spatial position information and the rotation information of the satellite are successfully decoupled by using a two-channel network, the spatial position information of the satellite is learned by using an improved Vision Transformer model, the satellite attitude information is learned by using a Hourglass network, the design of the two-channel network and the advantages of each channel network accord with the characteristics of the satellite attitude information, and the satellite attitude estimation precision is effectively provided.
Drawings
FIG. 1 is a flow diagram of the present invention;
FIG. 2 is a partial sample graph of a URSO satellite attitude estimation data set in accordance with an embodiment of the present invention;
Detailed Description
In order to make the technical scheme, advantages and purposes of the invention clearer, the technical scheme of the invention is further explained by combining a method flow described by a specific example and referring to the attached drawings.
The embodiment 1 of the invention relates to a two-channel satellite attitude estimation network based on deep learning, which is shown in figure 1 and comprises the following steps:
the method comprises the steps of manufacturing a satellite attitude estimation image data set, adopting a simulation image, marking attitude information of a satellite in the image and manufacturing the satellite attitude estimation image data set into a label, and rendering a satellite picture into a vivid space environment by using non Engine 4 software when manufacturing the data set, or using a URSO satellite attitude estimation data set manufactured by university of California scholars by using non Engine 4 software.
The method comprises the steps of dividing a satellite attitude estimation image data set into a training set, a verification set and a test set according to the proportion of 7:2:1, wherein the training set is mainly used for training a satellite attitude estimation model, the verification set is input into the model along with the training set and is mainly used for adjusting model hyper-parameters, and the test set is mainly used for evaluating the performance of the trained satellite attitude estimation model.
And (3) carrying out data preprocessing on the satellite attitude estimation image data set, firstly processing the satellite image by using an image preprocessing technology, and processing the image and the label into a data type capable of being input into a deep learning network.
Inputting a training set and a verification set subjected to data preprocessing into a ResNet model, extracting satellite image features by using components such as a convolution layer, a pooling layer and residual connection of the ResNet model, specifically extracting the satellite image features by using ResNet-18, and removing a global average pool layer and a last complete connection layer of an original network to keep the spatial feature resolution.
The improved Vision Transformer model is used for learning satellite spatial position information, and simultaneously a Hourglass network is used for learning satellite attitude information, wherein compared with the traditional Vision Transformer, the improved Vision Transformer firstly compresses and extracts a four-dimensional matrix into a three-dimensional matrix after processing ResNet model characteristics, namely [ batch, channel, length and width ] is compressed and extracted into [ batch, length and channel width ], so that the problem that the traditional Vision Transformer compresses the image length and width into a column to input the characteristics into a self-power mechanism to destroy the spatial position of an object in the image can be effectively avoided.
The network outputs the spatial position information and the rotation information of the satellite respectively, and three loss functions are required to be designed, wherein the three loss functions are as follows: calculating a space position information loss function of a satellite; calculating a loss function of the rotation information of the satellite body; thirdly, calculating an overall loss function, wherein the function expressions are respectively as follows:
Figure BDA0003596383410000051
Figure BDA0003596383410000061
L=αL pos +βL ori (α+β=1) (3)
l, L therein pos 、L ori Respectively representing a total loss function, a space position information loss function of the satellite and a satellite body rotation information loss function,
Figure BDA0003596383410000062
and
Figure BDA0003596383410000063
respectively a spatial position tag and a rotational tag,
Figure BDA0003596383410000064
and
Figure BDA0003596383410000065
is the spatial information and rotational information output by the model.
And reversely transmitting the three loss functions to the network, performing optimization iterative training by using a gradient descent method, and performing iterative training for multiple times to obtain an optimal satellite attitude estimation model.
Inputting the preprocessed test set into the satellite attitude estimation model to obtain attitude estimation information of the test set, and comparing the attitude estimation information with attitude information marked by the test set, thereby evaluating the satellite attitude estimation model.
Those skilled in the art will appreciate that the invention may be practiced without these specific details.

Claims (2)

1. A two-channel satellite attitude estimation network based on deep learning is characterized by mainly comprising the following steps:
step 1, constructing a satellite attitude estimation image data set, adopting a simulation image, and marking attitude information of a satellite in the image;
step 2, dividing the data set into a training set, a verification set and a test set according to a certain proportion;
step 3, satellite data preprocessing;
step 4, inputting the processed training set and the processed verification set into a ResNet model, and extracting image characteristics by using components of a convolution layer, a pooling layer, residual connection and the like of the ResNet model;
step 5, learning satellite spatial position information by using an improved Vision Transformer model, and learning satellite rotation information by using a Hourglass network;
step 6, the network respectively outputs the space position information and the attitude information of the satellite, calculates the distance between the output information and the marked attitude information, reversely transmits the distance information to the network, and performs optimization iterative training to obtain an optimal satellite attitude estimation model;
and 7, inputting the preprocessed test set into the satellite attitude estimation model to obtain attitude estimation information of the test set, and comparing the attitude estimation information with the attitude information marked by the test set, thereby evaluating the satellite attitude estimation model.
2. The deep learning based two-channel satellite attitude estimation network of claim 1, wherein:
in step 3, the improved Vision transform model is used to learn the satellite spatial position information, wherein the improved Vision transform is compared with the traditional Vision transform, after processing the characteristics of the ResNet model, the improved Vision transform firstly compresses and extracts the four-dimensional matrix into a three-dimensional matrix, namely [ batch, channel, length, width ] into [ batch, length, channel width ] so as to effectively avoid the problem that the traditional Vision transform compresses the image length and width into a column to input the characteristics into the self-attention mechanism, thereby destroying the spatial position of the object in the image.
CN202210393253.4A 2022-04-14 2022-04-14 Dual-channel satellite attitude estimation network based on deep learning Pending CN114842078A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210393253.4A CN114842078A (en) 2022-04-14 2022-04-14 Dual-channel satellite attitude estimation network based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210393253.4A CN114842078A (en) 2022-04-14 2022-04-14 Dual-channel satellite attitude estimation network based on deep learning

Publications (1)

Publication Number Publication Date
CN114842078A true CN114842078A (en) 2022-08-02

Family

ID=82565680

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210393253.4A Pending CN114842078A (en) 2022-04-14 2022-04-14 Dual-channel satellite attitude estimation network based on deep learning

Country Status (1)

Country Link
CN (1) CN114842078A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116051632A (en) * 2022-12-06 2023-05-02 中国人民解放军战略支援部队航天工程大学 Six-degree-of-freedom attitude estimation algorithm for double-channel transformer satellite

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110503689A (en) * 2019-08-30 2019-11-26 清华大学 Attitude prediction method, model training method and device
CN110969124A (en) * 2019-12-02 2020-04-07 重庆邮电大学 Two-dimensional human body posture estimation method and system based on lightweight multi-branch network
CN112183637A (en) * 2020-09-29 2021-01-05 中科方寸知微(南京)科技有限公司 Single-light-source scene illumination re-rendering method and system based on neural network
CN113095106A (en) * 2019-12-23 2021-07-09 华为数字技术(苏州)有限公司 Human body posture estimation method and device
CN113850151A (en) * 2021-09-02 2021-12-28 中山大学 Method, device, terminal and storage medium for identifying distraction behavior of driver
CN114332639A (en) * 2021-11-30 2022-04-12 中国人民解放军战略支援部队航天工程大学 Satellite attitude vision measurement algorithm of nonlinear residual error self-attention mechanism

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110503689A (en) * 2019-08-30 2019-11-26 清华大学 Attitude prediction method, model training method and device
CN110969124A (en) * 2019-12-02 2020-04-07 重庆邮电大学 Two-dimensional human body posture estimation method and system based on lightweight multi-branch network
CN113095106A (en) * 2019-12-23 2021-07-09 华为数字技术(苏州)有限公司 Human body posture estimation method and device
CN112183637A (en) * 2020-09-29 2021-01-05 中科方寸知微(南京)科技有限公司 Single-light-source scene illumination re-rendering method and system based on neural network
CN113850151A (en) * 2021-09-02 2021-12-28 中山大学 Method, device, terminal and storage medium for identifying distraction behavior of driver
CN114332639A (en) * 2021-11-30 2022-04-12 中国人民解放军战略支援部队航天工程大学 Satellite attitude vision measurement algorithm of nonlinear residual error self-attention mechanism

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
李孟锡等: "基于HOURGLASS网络语义关键点提取的光学图像空间目标姿态估计方法", 《信号处理》, vol. 37, no. 9, pages 1653 - 1662 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116051632A (en) * 2022-12-06 2023-05-02 中国人民解放军战略支援部队航天工程大学 Six-degree-of-freedom attitude estimation algorithm for double-channel transformer satellite
CN116051632B (en) * 2022-12-06 2023-12-05 中国人民解放军战略支援部队航天工程大学 Six-degree-of-freedom attitude estimation algorithm for double-channel transformer satellite

Similar Documents

Publication Publication Date Title
CN111160297B (en) Pedestrian re-identification method and device based on residual attention mechanism space-time combined model
CN108537742B (en) Remote sensing image panchromatic sharpening method based on generation countermeasure network
CN107506740B (en) Human body behavior identification method based on three-dimensional convolutional neural network and transfer learning model
CN110222591B (en) Lane line detection method based on deep neural network
CN113780149B (en) Remote sensing image building target efficient extraction method based on attention mechanism
CN110599537A (en) Mask R-CNN-based unmanned aerial vehicle image building area calculation method and system
CN114332639B (en) Satellite attitude vision measurement method of nonlinear residual error self-attention mechanism
CN114565655B (en) Depth estimation method and device based on pyramid segmentation attention
CN111681178B (en) Knowledge distillation-based image defogging method
CN114049356B (en) Method, device and system for detecting structure apparent crack
CN113011304A (en) Human body posture estimation method and system based on attention multi-resolution network
CN110210321B (en) Under-sample face recognition method based on multi-dimensional scale transformation network and block weighting method
CN112365434A (en) Unmanned aerial vehicle narrow passage detection method based on double-mask image segmentation
CN107766864B (en) Method and device for extracting features and method and device for object recognition
CN113112583B (en) 3D human body reconstruction method based on infrared thermal imaging
CN109461177A (en) A kind of monocular image depth prediction approach neural network based
CN106372597A (en) CNN traffic detection method based on adaptive context information
CN115131313A (en) Hyperspectral image change detection method and device based on Transformer
CN114842078A (en) Dual-channel satellite attitude estimation network based on deep learning
CN116958557A (en) Three-dimensional indoor scene semantic segmentation method based on residual impulse neural network
CN114743257A (en) Method for detecting and identifying image target behaviors
CN111223113B (en) Nuclear magnetic resonance hippocampus segmentation algorithm based on dual dense context-aware network
CN111709269A (en) Human hand segmentation method and device based on two-dimensional joint information in depth image
CN114140524B (en) Closed loop detection system and method for multi-scale feature fusion
Yang et al. UAV Landmark Detection Based on Convolutional Neural Network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20220802

WD01 Invention patent application deemed withdrawn after publication