CN115758707A - Modeling method, model and acquisition method of east-west retention strategy model of satellite - Google Patents
Modeling method, model and acquisition method of east-west retention strategy model of satellite Download PDFInfo
- Publication number
- CN115758707A CN115758707A CN202211410213.2A CN202211410213A CN115758707A CN 115758707 A CN115758707 A CN 115758707A CN 202211410213 A CN202211410213 A CN 202211410213A CN 115758707 A CN115758707 A CN 115758707A
- Authority
- CN
- China
- Prior art keywords
- satellite
- state
- value
- current
- state data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Feedback Control In General (AREA)
- Combined Controls Of Internal Combustion Engines (AREA)
Abstract
The invention relates to the field of aerospace, and provides a modeling method, a model, an obtaining method, equipment and a medium of a satellite east-west retention strategy model based on Double DQN, wherein the modeling method comprises the following steps: s1: acquiring a plurality of satellite training state data sets; s2: obtaining all tangential control behaviors and corresponding output Q values after the current moment; s3: obtaining the satellite state at the current moment and the tangential control behavior executed by the satellite; s4: obtaining the reward and the satellite state at the next moment; s5: storing the satellite combination state data set into an experience pool; s6: calculating a target value; s7: calculating an error; s8: updating the Q value; taking the satellite state at the next moment as the satellite state at the current moment; s9: repeating S3-S8 of the expected orbit control times, and updating the weight parameter of the target neural network; s10: and repeating S2-S9 until all the data of the satellite training state data set are input. The scheme can obtain an optimal decision strategy and reduce the consumption of satellite fuel.
Description
Technical Field
The invention relates to the technical field of aerospace, in particular to a modeling method, a model, an obtaining method, equipment and a medium of a satellite east-west retention strategy model.
Background
With the continuous development of human aerospace activities, more and more remote sensing satellites provide help for the daily life of people.
The GEO satellite is influenced by day and month gravitation and earth non-spherical perturbation in the operation process, so that drifting occurs in the east-west direction, and the maintaining control of the east-west position of the GEO three-axis stable satellite plays an important role in the aerospace field.
The method comprises the steps of firstly analyzing the satellite east-west drift caused by the influence of various shooting forces such as the earth spherical shape and the sunlight pressure on the satellite in the process of orbital operation, so as to cause the state change of the satellite, then modeling according to the shooting forces and formulating an east-west keeping strategy, further optimizing the keeping parameters and calculating the consumption of a propellant. In the prior art, various perturbation forces received by a satellite in an orbit operation process are subjected to complex modeling, however, due to the complexity of space stress and the uncertainty of parameters of the satellite, the satellite cannot be accurately modeled, the number of parameters is large, the calculation is complex, the accuracy of satellite object control is further influenced, and more fuel may be consumed.
Therefore, it is necessary to develop a modeling method, a model, an obtaining method, a device and a medium of a satellite east-west preservation policy model, so as to reduce the modeling difficulty and accurately calculate the east-west preservation policy.
Disclosure of Invention
The invention aims to provide a modeling method, a modeling model, an obtaining method, equipment and a medium of a satellite east-west keeping strategy model, which do not need to carry out complex modeling when carrying out east-west position keeping on a GEO triaxial stable satellite, do not need to consider the complexity of space stress and the uncertainty of the parameters of the satellite, have strong behavior decision-making capability in reinforcement learning, can obtain an optimal decision-making strategy and reduce the consumption of satellite fuel.
In order to solve the above technical problems, as an aspect of the present invention, a method for modeling a satellite east-west preservation policy model based on Double DQN is provided, including the following steps:
s1: initializing a model, and setting a plurality of groups of satellite training state data sets, wherein each group of satellite training state data sets comprises an initial state of a satellite, a plurality of expected orbit control moments and expected orbit control times; the initial state of each satellite comprises an initial time satellite state; satellite states include mean longitude and eccentricity vectors;
s2: inputting initial time satellite states of a group of satellite training state data sets into the model to obtain all tangential control behaviors after the initial time and corresponding output Q values;
s3: acquiring the state of the satellite at the current moment, and acquiring a tangential control behavior executed by the satellite according to a greedy strategy;
s4: executing a tangential control action to obtain the satellite state at the next moment; obtaining reward according to the satellite state and the east-west keeping strategy reward function at the next moment; the east-west keeping policy reward function adopts formula 1:
wherein r is t Reward, Δ R, for tangential control action on the satellite at the present moment t ＝R t+1 -R 0 ，Δe t ＝e t+1 -e 0 ，R t+1 The longitude flatness of the next moment to the current moment, e t+1 Eccentricity vector, R, at the next instant of the current instant 0 Flat longitude of nominal orbit, e 0 Is the eccentricity vector, Δ R, of the nominal track t Is the difference in flatness Δ e at the next instant of the current instant t The eccentricity vector difference at the next moment of the current moment; t is t 0 The expected orbit control time closest to the current time is obtained; t is the current moment;
s5: storing the satellite state at the current moment, the tangential control action executed by the satellite, the reward and the satellite state at the next moment into an experience pool as a group of satellite combination state data groups;
s6: taking out a plurality of satellite combination state data sets from the experience pool, and calculating the target value of each satellite combination state data set according to the target neural network weight parameter;
s7: calculating an error according to the loss function, and updating the weight parameter of the current neural network;
s8: updating the Q value according to the value function; taking the satellite state at the next moment as the satellite state at the current moment;
s9: repeating steps S3-S8, wherein the number of times of executing steps S3-S8 is equal to the expected orbit control number of times of the set of satellite training state data; after the steps S3-S8 of the appointed iteration times are executed repeatedly, updating the weight parameter of the target neural network according to the weight parameter of the current neural network;
s10: and repeatedly executing the steps S2-S9 until all the data of the satellite training state data set are input.
According to an exemplary embodiment of the invention, initializing the model in step S1 comprises defining a loss function.
According to an exemplary embodiment of the present invention, the input of the model is the satellite state, and the output is the return value (Q value) after the execution of the tangential control action by the satellite.
According to an example embodiment of the present invention, the satellite states include: mean longitude drift rate, eccentricity vector, tilt vector, and mean longitude. The mean longitude drift rate is denoted by L, the eccentricity vector by e, the inclination vector by i, and the mean longitude by R.
According to an exemplary embodiment of the present invention, in step S1, the satellite flat longitude is obtained according to formula 3:
wherein R represents the satellite mean longitude, a s Is the semi-major axis of the geostationary orbit, a 0 Is the orbital semi-major axis of the satellite;
the satellite eccentricity vector obtaining method adopts a formula 4: :
wherein e represents the eccentricity of the satellite, omega represents the right ascension of the satellite, omega represents the argument of the perigee; the two-dimensional eccentricity vector of the satellite in orbit is (e) x ，e y )。
According to an exemplary embodiment of the present invention, in step S3, during the first loop, the current satellite state is the initial satellite state.
According to an exemplary embodiment of the present invention, in step S3, the method for obtaining the tangential control behavior executed by the satellite according to the greedy policy includes: the satellite randomly selects a tangential control behavior according to a first specified probability or executes the tangential control behavior corresponding to the maximum Q value according to a second specified probability; the sum of the first specified probability and the second specified probability equals 1.
According to an exemplary embodiment of the present invention, in step S6, the method for calculating the target value of each satellite combination state data set according to the target neural network weight parameter uses formula 2:
wherein, y j Representing the target value, gamma is the discount value, theta' is the weight parameter of the target neural network, theta is the current godThrough the network weight parameter, the network weight is calculated,
represents the Q value obtained after the satellite performs the tangential control action a at the next moment in a group of satellite combination state data sets,
represents the tangential control action s corresponding to the maximum Q value obtained after the tangential control action a is executed by the satellite at the next moment in the combined state data set of the group of satellites j+1 Representing the satellite state at the next time in a set of satellite constellation state data sets, a representing the tangential control action performed by the satellite, r j Representing a reward in a set of satellite constellation state data sets.
According to an exemplary embodiment of the present invention, in step S7, the loss function adopts formula 5:
wherein, y j Representing the target value, theta is the current weight parameter of the neural network, Q(s) j ，a j (ii) a Theta) represents the current time satellite in a set of satellite combined state data sets performing a tangential control action a j Value of Q after, s j Representing the satellite state at the current time in a set of satellite constellation state data sets, a j Representing the tangential control action performed by the satellite at the current moment, and m is the number of the satellite combination state data sets.
According to an exemplary embodiment of the present invention, in step S8, the method for updating the Q value according to the value function adopts formula 6:
Q(s t ，a t )←Q(s t ，a t )+α[r t +γmax Q(s t+1 ，a t )-Q(s t ，a t )] (6)；
wherein, Q(s) at the left side of the arrow t ，a t ) The satellite representing the updated current time performs a tangential control action a t The value of Q after the process is finished,q(s) to the right of the arrow t ，a t ) The satellite representing the current moment before update performs a tangential control action a t Later Q value, Q(s) t+1 ，a t ) The satellite performs a tangential control action a at the next moment in time representing the current moment in time before updating t The latter Q value, alpha is the weight, gamma is the discount value, s t Representing the state of the satellite at the current time, a t Representing the tangential control action, s, performed by the satellite at the current moment t+1 The satellite state at the next moment in time, r, representing the current moment in time t Indicating the satellite state at the current time as s t Taking a tangential control action a t The reward earned later.
the time t is the current time, and the time t +1 is the next time of the current time.
The invention provides a satellite east-west preservation strategy model based on Double DQN, and the model is established by adopting the modeling method of the satellite east-west preservation strategy model based on Double DQN.
As a third aspect of the present invention, a method for obtaining an east-west satellite maintenance optimal strategy is provided, wherein a satellite east-west maintenance strategy model based on Double DQN is established by using the modeling method of the satellite east-west maintenance strategy model based on Double DQN;
obtaining an optimal strategy according to the model;
the method for obtaining the optimal strategy according to the model adopts a formula 7:
wherein, pi represents the strategy of tangential control of the satellite, pi * Represents the optimal tangential control strategy learned by the model, i.e. the satellite passes through the strategy pi under the condition that the satellite state is s at the initial moment * The control action a of (a) yields the greatest return.
As a fourth aspect of the present invention, there is provided an electronic apparatus comprising:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the method for modeling a Double DQN-based satellite east-west retention policy model.
As a fifth aspect of the present invention, a computer readable medium is provided, on which a computer program is stored, which when executed by a processor, implements the method for modeling a Double DQN-based east-west preservation policy model for satellites.
The invention has the beneficial effects that:
according to the scheme, the modeling is carried out through the neural network, the deep reinforcement learning and decision making are carried out by using the current satellite state data, the complex modeling is carried out without using various perturbation forces received by the satellite in the orbital operation process, the optimal east-west control strategy can be obtained, the consumption of satellite fuel can be reduced, and the method has important significance and value for practical aerospace application.
Drawings
Fig. 1 schematically shows a step diagram of a modeling method of a satellite east-west preservation policy model based on Double DQN.
Fig. 2 schematically shows a block diagram of an electronic device.
FIG. 3 schematically shows a block diagram of a computer-readable medium.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals denote the same or similar parts in the drawings, and thus, a repetitive description thereof will be omitted.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the subject matter of the present application can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the application.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
It will be understood that, although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one element from another. Thus, a first component discussed below could be termed a second component without departing from the teachings of the present concepts. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
It should be understood by those skilled in the art that the drawings are merely schematic representations of exemplary embodiments, and that the blocks or flowchart illustrations in the drawings are not necessarily required to practice the present application and, therefore, should not be considered to limit the scope of the present application.
According to the scheme, observation information is obtained from the environment based on strong perception capability of deep learning, and an expected return value is obtained based on strong decision-making capability of reinforcement learning to evaluate the action value. The entire learning process can be described as: at a certain moment, the satellite interacts with the flying environment to acquire observation information, the current state information is mapped into corresponding actions (control actions) through the neural network, the environment reacts to the actions to acquire corresponding reward values and next observation information, and the complete interaction information is stored in an experience pool. By continuously cycling the above processes, the optimal strategy for achieving the target can be finally obtained.
The satellite described in the scheme is a GEO triaxial stable satellite. Geosynchronous orbit (GEO), refers to a circular orbit of a satellite orbiting the earth at approximately 36000 kilometers above the equator of the earth. Because the orbit of a satellite is synchronized with the earth rotation about the earth, and the satellite and the earth are in a relatively stationary state, the satellite traveling in such an orbit is referred to as a "geostationary satellite" for short, and is also referred to as a "geostationary satellite" or a "stationary satellite". The three-axis stability means that the satellite does not rotate, and the body is stable in the X direction, the Y direction and the Z direction, in other words, a certain attitude relation is kept between the body and the earth.
A Deep Q network (Deep Q Networks, DQN for short) algorithm is a network in Deep reinforcement learning and is a combination of Deep learning and Q learning. The method integrates the advantages of reinforcement learning and deep learning, so that the method is widely applied to various fields at present.
Deep reinforcement learning is used as a new research hotspot in the field of artificial intelligence, and the deep learning and the reinforcement learning are combined, so that direct control and decision from original input to output are realized through an end-to-end learning mode. Because the deep learning is based on a neural network structure, the deep learning has stronger perception capability to the environment, but lacks certain decision control capability; and reinforcement learning has very strong behavior decision-making capability. Therefore, the perception capability of deep learning and the decision capability of the reinforcement learning are combined in the deep reinforcement learning, the advantages are complementary, and the control strategy can be directly learned from high-dimensional original data. Since the deep reinforcement learning method is provided, substantial breakthrough is achieved in a plurality of tasks requiring sensing of high-dimensional original input data and decision control, and due to the end-to-end learning advantage of deep learning, the problems of difficult modeling and difficult planning can be solved by deep reinforcement learning.
The DQN algorithm uses the same network for calculating the target value and the current value, i.e. the calculation of the target value uses the parameters in the Q network to be trained, and the target value is used to update the parameters of the network, so that the two depend on each other circularly, which is not favorable for the convergence of the algorithm. Compared with the DQN, the Nature DQN increases a target network, reduces the dependency relationship between the calculation of a target Q value and Q network parameters to be updated through a double-network structure, and integrates the advantages of reinforcement learning and deep learning, thereby greatly improving the stability of the DQN algorithm.
Nature DQN reduces the correlation between the target value of the computational target network and the current network parameters by using two independent but identical Q networks (one as the current Q network and the other as the target Q network). The target network is updated at regular intervals by copying the weight parameters of the current network to the target network, and the target Q value is kept unchanged in a period of time by the double-network structure, so that the correlation between the calculated target Q value and the current network parameters is reduced, and the convergence and the stability of the algorithm are improved.
When the DQN and the Nature DQN algorithms are used for optimizing the target by the value function, action selection and strategy evaluation are realized on the basis of the same value function. Neither DQN nor Nature DQN can overcome the inherent defect of Q-Learning, i.e., over-estimation, so that the estimated value function is larger than the true value.
Overestimation means that the estimated value function is larger than the true value function, and the root cause of the overestimation lies mainly in the maximization operation in Q-Learning, i.e. the target value
Where max is operated such that the estimated value function is larger than the true value of the ratio function (note: for a true policy and in a given situation, the action that maximizes the Q value is not chosen every time, since a true policy is generally a stochastic policy, where choosing the Q value that maximizes the action directly at the target value would result in a higher target value than the true value).
Different from the two algorithms, double DQN (DDQN for short) is based on a Nature DQN dual-network architecture, action selection and strategy evaluation are separately performed, the optimal action is selected by using the current neural network weight parameter theta, and the optimal action is evaluated by using the target neural network weight parameter theta', so that the problem of overestimation of the DQN and the Nature DQN algorithms is solved. The difference between the DDQN algorithm and the algorithm steps of the Nature DQN is different in the way the target Q value is calculated. The DDQN can estimate more accurate Q value, and more stable and effective strategy can be obtained.
As a first embodiment of the present invention, a method for modeling a satellite east-west preservation policy model based on Double DQN is provided, as shown in fig. 1, including the following steps:
s1: initializing a model, and setting a plurality of groups of satellite training state data sets, wherein each group of satellite training state data sets comprises an initial state of a satellite, a plurality of expected orbit control moments and expected orbit control times; the initial state of each satellite comprises an initial time satellite state; the satellite states include the mean longitude and the eccentricity vector.
The input of the model is the satellite state, and the output is the return value (Q value) after the execution of the tangential control action of the satellite.
The satellite states further include: mean longitude drift rate, tilt vector. The mean longitude drift rate is denoted by L, the eccentricity vector by e, the inclination vector by i, and the mean longitude by R.
The method for initializing the model comprises the following steps: defining a loss function; initializing the capacity of an experience pool to be N, wherein the experience pool is used for storing training samples; initializing a current neural network weight parameter theta and a target neural network weight parameter theta ', theta' = theta of the network model; the input of the initialized network is the satellite state s, and the calculated network output is the return value Q after the satellite executes the tangential control action.
And a plurality of groups of satellite training state data sets form a data set, the data of the satellite states in the data set is more than or equal to 100 groups, and the more the data of the satellite states, the more accurate the result trained by the model.
The data of the multiple groups of satellite training state data sets are data of a training set, and simulation data can be adopted, or simulation data and real data can be combined. The time line within a time period comprises a plurality of time points, the state of the satellite at each time point is different, and different effects can be obtained when the orbit control strategy is executed at different time points. According to the scheme, through a plurality of groups of satellite training state data groups, the satellite state of each group of satellites at the initial moment corresponds to the satellite state of a time point, the time points corresponding to the initial moments of each group of satellite training state data groups are different, namely the initial moments of each group of satellite training state data groups are different.
S2: and inputting the initial time satellite states of a group of satellite training state data groups into the model to obtain all tangential control behaviors after the initial time and the corresponding output Q values.
And after the initial moment satellite executes the tangential control action, the state of the next moment satellite is obtained. And after the satellite executes the tangential control action at the next moment, the satellite state at the next moment is obtained. And by analogy, the tangential control behaviors at a plurality of next moments are obtained.
S3: and acquiring the state of the satellite at the current moment, and acquiring a tangential control behavior executed by the satellite according to a greedy strategy.
And in the initial circulation, the satellite state at the current moment is the satellite state at the initial moment.
The method for acquiring the tangential control behavior executed by the satellite according to the greedy strategy comprises the following steps: the satellite randomly selects a tangential control behavior with a first specified probability or executes the tangential control behavior corresponding to the maximum Q value with a second specified probability; the sum of the first specified probability and the second specified probability equals 1.
If the first specified probability is larger than the second specified probability, the method for obtaining the tangential control behavior executed by the satellite according to the greedy strategy adopts the following steps: randomly selecting a tangential control behavior by the satellite with a first specified probability;
if the second designated probability is greater than the first designated probability, the method for obtaining the tangential control behavior executed by the satellite according to the greedy strategy adopts the following steps: the satellite executes the tangential control action corresponding to the maximum Q value according to a second specified probability;
if the first specified probability is equal to the second specified probability, then one of the methods for obtaining the tangential control behavior executed by the satellite according to the greedy strategy is selected: the satellite randomly selects the tangential control action with a first specified probability or executes the tangential control action corresponding to the maximum Q value with a second specified probability.
The greedy policy is an epsilon-greedy policy.
The first assigned probability is ε, which decreases as the number of iterations increases.
The tangential control action performed by the satellite at the current moment is a t 。
S4: executing a tangential control action to obtain the satellite state at the next moment; awarding is carried out according to satellite states and east-west keeping strategy awarding functions at the next moment.
The satellite drift in the east-west direction consists of two parts: one is spherical perturbation which causes the satellite longitude and latitude to drift, and the other is eccentricity perturbation generated by sunlight pressure which causes the satellite longitude and latitude periodic oscillation. East-west preservation of the satellite mainly includes flatness preservation and eccentricity preservation. The flatness maintenance mainly utilizes east-west maneuvering to correct the drift rate of the satellite, and one satellite is maintained in a narrow orbit window; in the satellite eccentricity vector keeping strategy, the sun pointing to the near place strategy is a more common method, and the method enables the mean eccentricity vector of the satellite to be always kept in an eccentricity control circle in a control period, and enables the average direction of a connecting line from the center of the control circle to the end point of the mean eccentricity vector to point to the sun direction.
In the two-body problem, the mean longitude (mean longitude) is a longitude value of an object moving on an imaginary circular orbit whose orbit inclination is 0.
Eccentricity (Eccentricity), i.e. Eccentricity, is a mathematical quantity used to describe the shape of a conic curve track. Defined as the ratio of the distance of the curve to the fixed point (focal point) to the distance to the fixed line (directrix). For an ellipse, eccentricity is the ratio of the distance between two foci (the focal length) to the length of the major axis. Eccentricity is generally indicated by e.
The satellite flat longitude obtaining method adopts a formula 3:
wherein R represents the satellite mean longitude, a s Is the semi-major axis of the geostationary orbit, a 0 Is the orbital semi-major axis of the satellite;
the satellite eccentricity vector obtaining method adopts a formula 4:
wherein e represents the satellite eccentricity vector, and the two-dimensional eccentricity vector of the satellite on the orbit is (e) x ，e y ) Omega represents the satellite ascension point right ascension and omega represents the argument of the perigee.
Both tangential thrust and radial thrust belong to in-plane maneuvers. Maneuvering using tangential thrust, known as tangential control (also known as east-west control or longitude control), can change the pan drift rate and eccentricity vector of the satellite. The maneuvering of radial thrust only changes the eccentricity vector of the satellite, but the radial thrust of the same magnitude can only achieve half the effect of tangential thrust. Tangential control is more efficient than radial control. Thus, satellite-borne east keeping maneuvers are mainly achieved by means of tangential thrust, and radial thrust is rarely used.
Tangential control refers to the maneuvering of a satellite in the orbital plane in the direction of velocity.
Because the tangential maneuvering can not only cause the change of the eccentricity vector, but also can cause the change of the orbit drift rate (semimajor axis), in order to optimize the ground measurement and control work and reduce the fuel consumption, the ideal state of things is kept to ensure that the satellite mean longitude keeping control period is the same as the eccentricity keeping period, and the optimum criterion of the in-plane maneuvering fuel consumption is utilized to keep and control the eccentricity vector and the mean longitude of the satellite.
The goal of the satellite east-west keeping strategy problem is to keep the satellite east-west position while minimizing fuel consumption as much as possible. The velocity impulses required for the east-west position holding control are mainly used for the holding control of the state.
The satellite control frequency is fixed (namely orbit control is carried out after a period of time), and the current orbit control quantity can ensure that the state of the satellite is in a keeping range in the next control. The velocity increment of the satellite at time t (a certain time) determines the state of the satellite orbit at extrapolated time t +1 (the time next to a certain time). For this purpose, a reward strategy at the time t is designed, that is, a reward function of the east-west keeping strategy is adopted, and the following formula 1 is adopted:
wherein r is t Reward, Δ R, obtained for tangential control action of the satellite at the current moment t ＝R t+1 -R 0 ，Δe t ＝e t+1 -e 0 ，R t+1 The longitude flatness of the next moment to the current moment, e t+1 Eccentricity vector, R, at the next instant of the current instant 0 Flat longitude of nominal orbit, e 0 Is the eccentricity vector, Δ R, of the nominal track t Is the difference in flatness Δ e at the next instant of the current instant t The eccentricity vector difference at the next moment of the current moment; t is t 0 The expected track control time closest to the current time is obtained; and t is the current moment. It is obvious that t | and | Δ e t The smaller the | the greater the prize won.
S5: and storing the satellite state at the current moment, the tangential control action executed by the satellite, the reward and the satellite state at the next moment into an experience pool as a group of satellite combination state data sets.
S6: and taking out a plurality of satellite combination state data sets from the experience pool, and calculating the target value of each satellite combination state data set according to the target neural network weight parameter.
The number of the satellite combination state data sets is m, m is a natural number larger than 0, and m is smaller than the number of the satellite training state data sets. The m groups of satellite combination state data groups are small-batch state data groups. The number of satellite constellation state data sets is determined from the data set of satellite training state data.
The method for calculating the target value of each satellite combination state data set according to the weight parameter of the target neural network adopts a formula 2:
wherein, y j Representing the target value, gamma is a discount value (attenuation factor), theta' is a target neural network weight parameter, theta is a current neural network weight parameter,
represents the Q value obtained after the satellite performs the tangential control action a at the next moment in a group of satellite combination state data sets,
represents the tangential control action s corresponding to the maximum Q value obtained after the satellite in the next time in the combined state data set of the group of satellites executes the tangential control action a j+1 Representing the satellite state at the next moment in a set of satellite combined state data sets, a representing the tangential control action performed by the satellite, r j A reward is indicated in a set of satellite constellation state data sets.
The equation 2 separates the action (tangential control action) selection and the strategy evaluation, selects the optimal action by using the current neural network weight parameter theta, and evaluates the optimal tangential control action by using the target neural network weight parameter theta', thereby solving the problem of over-estimation of the DQN and the Nature DQN algorithm.
And ending the task to finish the model convergence or iteration. When s is j+1 For model convergence or iteration to complete, y i Is equal to r j (ii) a When s j+1 When model convergence is not reached or iteration is completed, y i Is equal to
The conditions for model convergence are: the error calculated by the loss function is within a specified range.
The conditions for iteration completion are: all steps are executed.
S7: and calculating errors according to the loss function, and updating the current weight parameters of the neural network.
An error is also calculated based on the target value.
The loss function uses equation 5:
wherein, y j Representing the target value, theta is the current weight parameter of the neural network, Q(s) j ，a j (ii) a Theta) represents the current time satellite in a set of satellite combined state data sets performing a tangential control action a j Value of later Q, s j Representing the satellite state at the current time in a set of satellite constellation state data sets, a j Representing the tangential control action performed by the satellite at the current moment, r j Representing a reward in a set of satellite constellation state data sets; and m is the number of the satellite combination state data sets.
The error is the calculation result of the loss function by using the formula 5.
The current neural network weight parameters are updated by a Stochastic Gradient Descent (SGD) method.
r t 、a t 、s t 、s t+1 Samples in the data set representing the satellite training state data set, r j 、a j 、s j 、s j+1 Representing samples in an experience pool.
S5-S7 adjust the parameters of the model, so that the calculation accuracy of the model can be higher.
S8: updating the Q value according to the value function; and taking the satellite state at the next moment as the satellite state at the current moment.
The method for updating the Q value according to the value function uses equation 6:
Q(s t ，a t )←Q(s t ，a t )+α[r t +γmax Q(s t+1 ，a t )-Q(s t ，a t )] (6)；
wherein, Q(s) at the left side of the arrow t ，a t ) The satellite representing the updated current time performs a tangential control action a t The latter Q value, Q(s) on the right side of the arrow t ，a t ) The satellite representing the current moment before update performs a tangential control action a t Later Q value, Q(s) t+1 ，a t ) The satellite representing the next moment in time to the current moment in time before the update performs a tangential control action a t The latter Q value, α is the weight, γ is the discount value (attenuation factor), s t Representing the state of the satellite at the current time, a t Representing the tangential control action performed by the satellite at the current moment, s t+1 The satellite state at the next time representing the current time, r t Indicating the satellite state at the current time as s t Taking a tangential control action a t The reward obtained later.
Wherein both α and γ range between 0 and 1.
S9: repeating steps S3-S8, wherein the times for executing steps S3-S8 is equal to the expected orbit control times of the set of satellite training state data; and after the steps S3-S8 of the appointed iteration times are repeatedly executed, updating the weight parameter of the target neural network according to the current weight parameter of the neural network.
And updating the target neural network weight parameter to the current neural network weight parameter after finishing the iteration of the specified iteration times.
S10: and repeatedly executing the steps S2-S9 until all the data of the satellite training state data set are input.
According to the modeling method, satellite state data are used as input of a neural network model, a generated return value is used as output, a Double DQN neural network is adopted, complex modeling is carried out without various perturbation forces applied to a satellite in an orbital operation process, deep reinforcement learning is directly adopted for learning and decision making, improvement is carried out on the basis of a DQN algorithm, the method is suitable for training a large-scale neural network, the stability of the DQN algorithm is greatly improved, an optimal east-west control strategy can be obtained, and consumption of satellite fuel can be reduced, so that the method has important significance and value for practical aerospace application.
According to a second embodiment of the invention, the invention provides a satellite east-west preservation strategy model based on Double DQN, and the model is established by adopting the modeling method of the satellite east-west preservation strategy model based on Double DQN of the first embodiment.
According to a third embodiment of the invention, the invention provides a method for acquiring satellite east-west retention optimal strategies, which comprises the steps of establishing a satellite east-west retention strategy model based on Double DQN by adopting the modeling method of the satellite east-west retention strategy model based on Double DQN of the first embodiment;
and obtaining an optimal strategy according to the model.
The method for obtaining the optimal strategy according to the model adopts a formula 7:
wherein, pi represents the strategy of tangential control of the satellite, pi * Representing the optimal tangential control strategy learned by the model, i.e. satellite-like at the initial momentPassing strategy pi under the condition of s state * Produces the greatest return under tangential control action a.
According to a fourth embodiment of the present invention, there is provided an electronic device, as shown in fig. 2, and fig. 2 is a block diagram of an electronic device according to an exemplary embodiment.
An electronic device 200 according to this embodiment of the present application is described below with reference to fig. 2. The electronic device 200 shown in fig. 2 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in FIG. 2, electronic device 200 is embodied in the form of a general purpose computing device. The components of the electronic device 200 may include, but are not limited to: at least one processing unit 210, at least one memory unit 220, a bus 230 connecting different system components (including the memory unit 220 and the processing unit 210), a display unit 240, and the like.
Wherein the storage unit stores program code executable by the processing unit 210 to cause the processing unit 210 to perform the steps according to various exemplary embodiments of the present application described in the present specification. For example, the processing unit 210 may perform the steps as shown in fig. 1.
The storage unit 220 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM) 2201 and/or a cache memory unit 2202, and may further include a read only memory unit (ROM) 2203.
The storage unit 220 may also include a program/utility 2204 having a set (at least one) of program modules 2205, such program modules 2205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 200 may also communicate with one or more external devices 200' (e.g., keyboard, pointing device, bluetooth device, etc.), devices that enable a user to interact with the electronic device 200, and/or any device (e.g., router, modem, etc.) that the electronic device 200 may communicate with one or more other computing devices. Such communication may occur through input/output (I/O) interfaces 250. Also, the electronic device 200 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 260. The network adapter 260 may communicate with other modules of the electronic device 200 via the bus 230. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware.
Thus, according to a fifth embodiment of the present invention, there is provided a computer readable medium. As shown in fig. 3, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to make a computing device (which can be a personal computer, a server, or a network device, etc.) execute the above-mentioned method according to the embodiment of the present invention.
The software product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In situations involving remote computing devices, the remote computing devices may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to external computing devices (e.g., through the internet using an internet service provider).
The computer-readable medium carries one or more programs which, when executed by a device, cause the computer-readable medium to carry out the functions of the first embodiment.
Those skilled in the art will appreciate that the modules described above may be distributed in the apparatus according to the description of the embodiments, or may be modified accordingly in one or more apparatuses unique from the embodiments. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiment of the present invention.
The present invention has been described in terms of the preferred embodiment, and it is not intended to be limited to the embodiment. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A modeling method of a satellite east-west preservation policy model based on Double DQN is characterized by comprising the following steps:
s1: initializing a model, and acquiring a plurality of groups of satellite training state data sets, wherein each group of satellite training state data set comprises an initial state of a satellite, a plurality of expected orbit control moments and expected orbit control times; the initial state of each satellite comprises an initial time satellite state; satellite states include flat longitude and eccentricity vectors;
s2: inputting initial time satellite states of a group of satellite training state data sets into the model to obtain all tangential control behaviors after the initial time and corresponding output Q values;
s3: acquiring the state of the satellite at the current moment, and acquiring a tangential control behavior executed by the satellite according to a greedy strategy;
s4: executing a tangential control action to obtain the satellite state at the next moment; awarding according to the satellite state and east-west keeping strategy awarding function at the next moment; the east-west retention policy reward function adopts formula 1:
wherein r is t Reward, Δ R, obtained for tangential control action of the satellite at the current moment t ＝R t+1 -R 0 ，Δe t ＝e t+1 -e 0 ，R t+1 The longitude flatness of the next moment to the current moment, e t+1 Eccentricity vector, R, at the next instant of the current instant 0 Flat longitude of nominal orbit, e 0 Is the eccentricity vector, Δ R, of the nominal track t Is the difference in flatness Δ e at the next instant of the current instant t The eccentricity vector difference at the next moment of the current moment; t is t 0 The expected track control time closest to the current time is obtained; t is the current time;
s5: storing the satellite state at the current moment, the tangential control action executed by the satellite, the reward and the satellite state at the next moment into an experience pool as a group of satellite combination state data groups;
s6: taking out a plurality of satellite combination state data sets from the experience pool, and calculating the target value of each satellite combination state data set according to the target neural network weight parameter;
s7: calculating an error according to the loss function, and updating the weight parameter of the current neural network;
s8: updating the Q value according to the value function; taking the satellite state at the next moment as the satellite state at the current moment;
s9: repeating steps S3-S8, wherein the number of times of executing steps S3-S8 is equal to the expected orbit control number of times of the set of satellite training state data; after the steps S3-S8 of the appointed iteration times are repeatedly executed, updating the weight parameter of the target neural network according to the weight parameter of the current neural network;
s10: and repeatedly executing the steps S2-S9 until all the data of the satellite training state data set are input.
2. The modeling method for satellite east-west preservation policy model based on Double DQN of claim 1, wherein in step S1, the satellite mean longitude is obtained according to equation 3:
wherein, R represents the longitude of the satellite, as is the semi-major axis of the geostationary orbit, a0 is the semi-major axis of the orbit of the satellite;
the satellite eccentricity vector obtaining method adopts a formula 4:
wherein e represents the eccentricity of the satellite, omega represents the ascension of the satellite at the intersection point, and omega represents the argument of the near place; the two-dimensional eccentricity vector of the satellite in orbit is (e) x ，e y )。
3. The modeling method for satellite east-west preservation policy model based on Double DQN of claim 1, wherein in step S3, the method for obtaining the tangential control behavior executed by the satellite according to greedy policy comprises: the satellite randomly selects a tangential control behavior according to a first specified probability or executes the tangential control behavior corresponding to the maximum Q value according to a second specified probability; the sum of the first specified probability and the second specified probability equals 1.
4. The method for modeling the east-west preservation policy model of satellite based on Double DQN of claim 1, wherein in step S6, the method for calculating the target value of each satellite combined state data set according to the target neural network weight parameter uses equation 2:
wherein, y j Representing the target value, gamma is the discount value, theta' is the target neural network weight parameter, theta is the current neural network weight parameter,
representing the Q value obtained after the satellite performs the tangential control action a at the next moment in a set of satellite combined state data sets,
represents the tangential control action s corresponding to the maximum Q value obtained after the tangential control action a is executed by the satellite at the next moment in the combined state data set of the group of satellites j+1 Representing the satellite state at the next time in a set of satellite constellation state data sets, a representing the tangential control action performed by the satellite, r j Representing a reward in a set of satellite constellation state data sets.
5. The method for modeling the satellite east-west preservation policy model based on Double DQN of claim 1, wherein in step S7, the loss function is represented by equation 5:
wherein, y j Representing the target value, theta is the current weight parameter of the neural network, Q(s) j ，a j (ii) a Theta) represents the current time satellite in a set of satellite combined state data sets performing the tangential control action a j Value of Q after, s j Representing the current time satellite shape in a set of satellite constellation state data setsState a of j Representing the tangential control action performed by the satellite at the current moment, and m is the number of the satellite combined state data sets.
6. The method for modeling a east-west preservation policy model of Double DQN-based satellite according to claim 1, wherein in step S8, the method for updating Q value according to value function adopts formula 6:
Q(s t ，a t )←Q(s t ，a t )+α[r t +γmaxQ(s t+1 ，a t )-Q(s t ，a t )] (6)；
wherein, Q(s) at the left side of the arrow t ，a t ) Representing the updated current time satellite performing the tangential control action a t The latter Q value, Q(s) on the right side of the arrow t ，a t ) The satellite representing the current moment before update performs a tangential control action a t Later Q value, Q(s) t+1 ，a t ) The satellite executes the tangential control action a at the next moment of time representing the current moment before updating t The latter Q value, alpha is the weight, gamma is the discount value, s t Representing the state of the satellite at the current time, a t Representing the tangential control action, s, performed by the satellite at the current moment t+1 The satellite state at the next time representing the current time, r t Indicating the satellite state at the current time as s t Taking a tangential control action a t The reward earned later.
7. A satellite east-west preservation policy model based on Double DQN, characterized in that the model is built using the modeling method of any one of claims 1-6.
8. A method for acquiring satellite east-west preservation optimal strategies, which is characterized in that a satellite east-west preservation strategy model based on Double DQN is established according to the modeling method of any one of claims 1-6;
obtaining an optimal strategy according to the model;
the method for obtaining the optimal strategy according to the model adopts a formula 7:
wherein, pi represents the strategy of tangential control of the satellite, pi * Represents the optimal tangential control strategy learned by the model, i.e. the satellite passes through the strategy pi under the condition that the satellite state is s at the initial moment * Produces the greatest return under tangential control action a.
9. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-6.
10. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211410213.2A CN115758707B (en) | 2022-11-10 | 2022-11-10 | Modeling method, system and acquisition method of satellite east-west maintenance strategy model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211410213.2A CN115758707B (en) | 2022-11-10 | 2022-11-10 | Modeling method, system and acquisition method of satellite east-west maintenance strategy model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115758707A true CN115758707A (en) | 2023-03-07 |
| CN115758707B CN115758707B (en) | 2023-05-12 |
Family
ID=85369390
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211410213.2A Active CN115758707B (en) | 2022-11-10 | 2022-11-10 | Modeling method, system and acquisition method of satellite east-west maintenance strategy model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115758707B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117744753A (en) * | 2024-02-19 | 2024-03-22 | 浙江同花顺智能科技有限公司 | Method, device, equipment and medium for determining prompt word of large language model |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20110067897A (en) * | 2009-12-15 | 2011-06-22 | 한국전자통신연구원 | Method for east-west station-keeping maneuver of geostationary orbit satellite |
| CN113258988A (en) * | 2021-05-13 | 2021-08-13 | 重庆邮电大学 | DQN-based multi-service low-orbit satellite resource allocation method |
| CN113378290A (en) * | 2021-05-12 | 2021-09-10 | 北京航空航天大学 | Orbit maintaining method for ultra-low orbit satellite |
| CN113613301A (en) * | 2021-08-04 | 2021-11-05 | 北京航空航天大学 | Air-space-ground integrated network intelligent switching method based on DQN |
| CN114362810A (en) * | 2022-01-11 | 2022-04-15 | 重庆邮电大学 | Low-orbit satellite beam hopping optimization method based on migration depth reinforcement learning |
| CN114967453A (en) * | 2022-05-25 | 2022-08-30 | 北京理工大学 | Satellite east-west coordination state initial value estimation method based on neural network |
-
2022
- 2022-11-10 CN CN202211410213.2A patent/CN115758707B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20110067897A (en) * | 2009-12-15 | 2011-06-22 | 한국전자통신연구원 | Method for east-west station-keeping maneuver of geostationary orbit satellite |
| CN113378290A (en) * | 2021-05-12 | 2021-09-10 | 北京航空航天大学 | Orbit maintaining method for ultra-low orbit satellite |
| CN113258988A (en) * | 2021-05-13 | 2021-08-13 | 重庆邮电大学 | DQN-based multi-service low-orbit satellite resource allocation method |
| CN113613301A (en) * | 2021-08-04 | 2021-11-05 | 北京航空航天大学 | Air-space-ground integrated network intelligent switching method based on DQN |
| CN114362810A (en) * | 2022-01-11 | 2022-04-15 | 重庆邮电大学 | Low-orbit satellite beam hopping optimization method based on migration depth reinforcement learning |
| CN114967453A (en) * | 2022-05-25 | 2022-08-30 | 北京理工大学 | Satellite east-west coordination state initial value estimation method based on neural network |
Non-Patent Citations (2)
| Title |
|---|
| 杨大林;徐波;高有涛;: "地球轨道卫星电推进变轨控制方法", 宇航学报 * |
| 马煦;孙海燕;黄显安;李永财;: "采用偏心率和倾角矢量联合隔离法实现双星共位", 电讯技术 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117744753A (en) * | 2024-02-19 | 2024-03-22 | 浙江同花顺智能科技有限公司 | Method, device, equipment and medium for determining prompt word of large language model |
| CN117744753B (en) * | 2024-02-19 | 2024-05-03 | 浙江同花顺智能科技有限公司 | Method, device, equipment and medium for determining prompt word of large language model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115758707B (en) | 2023-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chan et al. | Autonomous imaging and mapping of small bodies using deep reinforcement learning | |
| Zuo et al. | A case learning-based differential evolution algorithm for global optimization of interplanetary trajectory design | |
| Lagona et al. | Autonomous trajectory optimisation for intelligent satellite systems and space traffic management | |
| Little et al. | Space situational awareness sensor tasking: comparison of machine learning with classical optimization methods | |
| US11546001B2 (en) | Preprocessor for device navigation | |
| Wang et al. | Deep reinforcement learning-based autonomous mission planning method for high and low orbit multiple agile Earth observing satellites | |
| CN115758707A (en) | Modeling method, model and acquisition method of east-west retention strategy model of satellite | |
| Bonasera et al. | Designing impulsive station-keeping maneuvers near a sun-earth l2 halo orbit via reinforcement learning | |
| Gu et al. | Large region targets observation scheduling by multiple satellites using resampling particle swarm optimization | |
| CN115795816B (en) | Modeling method, model and acquisition method of satellite east-west maintenance strategy model | |
| CN115795817B (en) | Modeling method, system and acquisition method of satellite east-west maintenance strategy model | |
| Fraser | Adaptive extended Kalman filtering strategies for autonomous relative navigation of formation flying spacecraft | |
| Harvey et al. | Machine learning applications for sensor tasking with non-linear filtering | |
| Herrmann et al. | Autonomous spacecraft tasking using monte carlo tree search methods | |
| CN115758706B (en) | Modeling method, model and acquisition method of satellite east-west maintenance strategy model | |
| CN115806060B (en) | Modeling method, model and acquisition method of satellite relative phase maintaining strategy model | |
| CN115758705B (en) | Modeling method, system and acquisition method for satellite north-south maintenance strategy model | |
| CN115806062B (en) | Modeling method, system and acquisition method of satellite relative phase maintaining strategy model | |
| CN115892516B (en) | Modeling method, model and acquisition method of satellite relative phase maintaining strategy model | |
| CN115865167A (en) | Modeling method, model and acquisition method of satellite north-south conservation strategy model | |
| CN115806061B (en) | Modeling method, model and acquisition method of satellite relative phase maintaining strategy model | |
| Harvey et al. | Bridging heuristic and deep learning approaches to sensor tasking | |
| CN115865166A (en) | Modeling method, model and acquisition method of satellite north-south conservation strategy model | |
| CN115758704B (en) | Modeling method, system and acquisition method for satellite north-south maintenance strategy model | |
| Stephenson et al. | Reinforcement Learning For Earth-Observing Satellite Autonomy With Event-Based Task Intervals |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |