CN114115304A - Aircraft four-dimensional climbing track planning method and system - Google Patents

Aircraft four-dimensional climbing track planning method and system Download PDF

Info

Publication number
CN114115304A
CN114115304A CN202111248553.5A CN202111248553A CN114115304A CN 114115304 A CN114115304 A CN 114115304A CN 202111248553 A CN202111248553 A CN 202111248553A CN 114115304 A CN114115304 A CN 114115304A
Authority
CN
China
Prior art keywords
aircraft
dimensional
flight
climbing track
climbing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111248553.5A
Other languages
Chinese (zh)
Inventor
张洪海
周锦伦
万俊强
吕文颖
钟罡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Aeronautics and Astronautics
Original Assignee
Nanjing University of Aeronautics and Astronautics
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Aeronautics and Astronautics filed Critical Nanjing University of Aeronautics and Astronautics
Priority to CN202111248553.5A priority Critical patent/CN114115304A/en
Publication of CN114115304A publication Critical patent/CN114115304A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/08—Control of attitude, i.e. control of roll, pitch, or yaw
    • G05D1/0808—Control of attitude, i.e. control of roll, pitch, or yaw specially adapted for aircraft
    • G05D1/0816—Control of attitude, i.e. control of roll, pitch, or yaw specially adapted for aircraft to ensure stability
    • G05D1/0833—Control of attitude, i.e. control of roll, pitch, or yaw specially adapted for aircraft to ensure stability using limited authority control

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention discloses an aircraft four-dimensional climbing track planning method and system, which are used for obtaining an optional action space in a flight state according to the performance of an aircraft, solving the optimal action in the climbing of the aircraft by adopting a reinforced learning algorithm of a single-step time sequence difference updating evaluation strategy, fitting a climbing track obtained based on the optimal action to obtain an aircraft four-dimensional climbing track, and realizing the high-efficiency and refined aircraft four-dimensional climbing track planning.

Description

Aircraft four-dimensional climbing track planning method and system
Technical Field
The invention relates to a method and a system for planning a four-dimensional climbing flight path of an aircraft, and belongs to the field of flight path planning.
Background
Since the introduction of the concept of Trajectory Based Operation (TBO), four-dimensional Trajectory planning has been rapidly developed as one of its key technologies. The traditional path planning methods such as an A-star algorithm, an ant colony algorithm, a genetic algorithm and the like have complex processing flows on the aircraft time sequence difference, so that the path planning methods are difficult to take advantages in the four-dimensional climbing track planning, and therefore a new aircraft four-dimensional climbing track planning method is urgently needed.
Disclosure of Invention
The invention provides a method and a system for planning a four-dimensional climbing track of an aircraft, which solve the problems disclosed in the background technology.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
an aircraft four-dimensional climbing track planning method comprises the following steps:
acquiring optional actions in various flight states when an aircraft climbs in a simulated flight experiment;
according to the optional actions in the flight state, aiming at the minimum oil consumption and the minimum running time of the cruise waypoints, adopting a reinforced learning algorithm of a single-step time sequence difference updating evaluation strategy to solve the optimal actions of the aircraft in climbing;
and fitting the climbing track obtained based on the optimal action to obtain the four-dimensional climbing track of the aircraft.
The flight conditions include horizontal position, altitude position, velocity, and pitch angle, and the selectable actions include acceleration and pitch angle velocity.
Interactive models in the reinforcement learning algorithm comprise an environment model and an intelligent agent model;
an environment model: calculating the oil consumption and the position change of the aircraft in unit time of operation in the current flight state according to the parameters of the environment where the aircraft is located; calculating an awarding value given by the environment according to the oil consumption and the position change, and taking the awarding value as a parameter for evaluating the current flight state of the aircraft;
the intelligent agent model comprises: an optimal action is selected from the selectable actions.
The reward value calculation formula is as follows:
Figure BDA0003321642660000021
wherein, r(s)t,at) For aircraft in time sequence t, flight state stLower selection action atThe reward value given by the environment;
rt 1obtaining a positive reward value for the aircraft approaching a cruise waypoint;
rt 2obtaining a negative reward value for the aircraft after t time sequence;
rt 3acting on aircraft atFuel is consumed and a negative prize value is achieved.
Fitting the climbing track obtained based on the optimal action to obtain the four-dimensional climbing track of the aircraft, wherein the four-dimensional climbing track comprises the following steps:
determining the conversion height and the flat flying distance of the aircraft when the stairs climb according to the performance of the aircraft;
and fitting the climbing track obtained based on the optimal action according to the converted height and the plane flight distance to obtain the four-dimensional climbing track of the aircraft.
An aircraft four-dimensional climb trajectory planning system comprising:
an acquisition module: acquiring optional actions in various flight states when an aircraft climbs in a simulated flight experiment;
a reinforcement learning module: according to the optional actions in the flight state, aiming at the minimum oil consumption and the minimum running time of the cruise waypoints, adopting a reinforced learning algorithm of a single-step time sequence difference updating evaluation strategy to solve the optimal actions of the aircraft in climbing;
a track fitting module: and fitting the climbing track obtained based on the optimal action to obtain the four-dimensional climbing track of the aircraft.
The flight conditions include horizontal position, altitude position, velocity, and pitch angle, and the selectable actions include acceleration and pitch angle velocity.
Interactive models in the reinforcement learning algorithm comprise an environment model and an intelligent agent model;
an environment model: calculating the oil consumption and the position change of the aircraft in unit time of operation in the current flight state according to the parameters of the environment where the aircraft is located; calculating an awarding value given by the environment according to the oil consumption and the position change, and taking the awarding value as a parameter for evaluating the current flight state of the aircraft;
the intelligent agent model comprises: an optimal action is selected from the selectable actions.
The reward value calculation formula is as follows:
r(st,at)＝rt 1+rt 2+rt 3
wherein, r(s)t,at) For aircraft in time sequence t, flight state stLower selection action atThe reward value given by the environment;
rt 1obtaining a positive reward value for the aircraft approaching a cruise waypoint;
rt 2obtaining a negative reward value for the aircraft after t time sequence;
rt 3acting on aircraft atFuel is consumed and a negative prize value is achieved.
The track fitting module comprises:
a conversion height level flight distance determination module: determining the conversion height and the flat flying distance of the aircraft when the stairs climb according to the performance of the aircraft;
a fitting module: and fitting the climbing track obtained based on the optimal action according to the converted height and the plane flight distance to obtain the four-dimensional climbing track of the aircraft.
The invention achieves the following beneficial effects: according to the method, the optional action space in the flight state is obtained according to the performance of the aircraft, the reinforced learning algorithm of the single-step time sequence difference updating evaluation strategy is adopted, the optimal action of the aircraft in climbing is solved, the climbing track obtained based on the optimal action is fitted, the aircraft four-dimensional climbing track is obtained, and the high-efficiency and refined aircraft four-dimensional climbing track planning is realized.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a schematic diagram of aircraft state-motion space one-hot code discretization;
FIG. 3 is a line graph of the average total round award values per 100 rounds of learning by the agent for 50 ten thousand learning sessions;
FIG. 4 is a line graph of the average total round award values per 1000 rounds of learning by the agent for 50 ten thousand learning sessions;
FIG. 5 is a schematic diagram of the effect of track fitting.
Detailed Description
The invention is further described below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present invention is not limited thereby.
As shown in fig. 1, a method for planning a four-dimensional climbing track of an aircraft includes the following steps:
step 1, acquiring optional actions in various flight states when an aircraft climbs in a simulated flight experiment; the simulated flight experiment is a simulated flight experiment developed based on the real performance of the aircraft;
step 2, according to the optional actions in the flight state, aiming at the minimum fuel consumption and the minimum running time of the cruise waypoints, adopting a single-step time sequence difference to update a reinforcement learning algorithm of an evaluation strategy, and solving the optimal actions of the aircraft during climbing;
and 3, fitting the climbing track obtained based on the optimal action to obtain the four-dimensional climbing track of the aircraft.
According to the method, the optional action space in the flight state is obtained according to the performance of the aircraft, the reinforced learning algorithm of the single-step time sequence difference updating evaluation strategy is adopted, the optimal action of the aircraft in climbing is solved, the climbing track obtained based on the optimal action is fitted, the aircraft four-dimensional climbing track is obtained, and the high-efficiency and refined aircraft four-dimensional climbing track planning is realized.
The flight state of the aircraft is generally obtained by a Flight Management System (FMS) and sent to a pilot dashboard and a controller secondary radar or ADS-B device, so it is not difficult to obtain the flight attitude in the flight path planning. In the simulated flight experiment, pilot operation can be simulated according to the real performance of the aircraft, and the corresponding flight state of the aircraft is obtained.
Aircraft position, speed, etc. are continuous variables, and to derive a limited number of features from a continuous space, one-hot coding (one-hot coding) may be used to process the continuous flight state of the aircraft into discrete flight states, i.e., s (x)i-δ≤xi≤xi+δ)＝siAnd δ is the coding granularity size.
The flight state includes horizontal position, altitude position, velocity and pitch angle, and the selectable actions include acceleration and pitch angle velocity, which may be specifically denoted as si＝(xi,yi,vi,θi)，ai＝(acci,wi)，si∈S′，aiE.g. A wherein siFor i-time flight status of the aircraft, xi、yi、vi、θiRespectively representing horizontal position, altitude position, velocity and pitch angle, aiIs s isiOne optional action next, acci、wiRespectively representing acceleration and pitch rate, S' being the set of all flight states, a being the set of all optional actions (see fig. 2)
Due to the aerodynamic properties of fixed wing aircraft, the aircraft horizontal velocity does not have to be 0. In order to facilitate algorithm design and accelerate algorithm iterative convergence speed, the aircraft flight state space after discretization should meet the following conditions: the flight state of the aircraft after a unit time changes in a discrete space (dynamic) and still belongs to the discrete space (closed).
Should avoid the occurrence of si+1＝si＝V(si|ai) Or
Figure BDA0003321642660000051
In the case of (1), i.e. requiring si+1＝V(si|ai),si+1≠siAnd si+1＝V(ai|si),si+1E S', is solved by setting a minimum speed, i.e. the coding granularity needs to be such that running the aircraft for one second at the minimum speed will also cause a change in the horizontal position status. Meanwhile, the decision should be based on the state flight state of the aircraft one-hot code; wherein V(s)i|ai) Meaning that the aircraft is in flight siTime selection action aiTransition to the next time-sequence flight State si+1The state transition equation of (1).
A plurality of optional actions are generally generated in each flight state, the flight state and the optional actions are used as the input of a reinforcement learning algorithm of a single-step time sequence difference updating evaluation strategy, and the optimal actions in the flight state can be obtained through iterative calculation.
The reinforcement learning algorithm of the single-step time sequence differential updating evaluation strategy generally has two interaction models including an environment model and an intelligent agent model.
An environment model: according to parameters (height, atmospheric temperature and the like) of the environment where the aircraft is located, an aircraft performance model and an oil consumption calculation model are built, and the oil consumption and position change of the aircraft in unit time of operation in the current flight state are calculated; and calculating the reward value given by the environment according to the oil consumption and the position change, and taking the reward value as a parameter for evaluating the current flight state of the aircraft, namely evaluating the quality of the current flight state.
The set prize system and prize value calculation may be as follows:
1. reward and penalty system
The aircraft approaches the cruising waypoint and obtains a certain positive reward value rt 1(so that the decision is biased to fly to the destination); the aircraft obtains a certain negative reward value r after t time sequencet 2(reduced flight time); aircraft acting atConsuming fuel oil and obtaining a certain negative reward value rt 3(reduce fuel consumption).
2. Therefore, based on the above-mentioned reward system, the formula for calculating the reward value may be as follows:
r(st,at)＝rt 1+rt 2+rt 3
wherein, r(s)t,at) For aircraft in time sequence t, flight state stLower selection action atThe environment then awards the prize value.
The intelligent agent model comprises: the intelligent agent is used as a decision maker for selecting the flight action of the aircraft, and the optimal action is selected from the selectable actions and executed.
Based on the performance of the aircraft, optional actions of the aircraft in different flight states can be acquired, specifically as follows:
1) determining a specific aircraft model and a speed interval corresponding to the flight height based on an aircraft performance database:
vmin≤vsi+accai≤vmax
vmin＝CVmin×Vstall
vmax＝VMO/MMO
wherein v issiFor the current i time sequence aircraft speed, accaiRepresenting the acceleration decisions available in the current aircraft flight state, vmax、vminRespectively representing maximum and minimum steerable speeds, C, under aeronautical performance constraintsVmin、VMO、MMOAll are corresponding performance super-parameters of a specific machine type in a BADA model, wherein VMORepresents the maximum steerable corrected airspeed, VstallFor aircraft stall speed, MMORepresenting the maximum manageable mach number.
2) Determining the acceleration interval corresponding to the specific aircraft model and the flight height based on the aircraft performance database:
0≤accai≤amax
Thrmaxclimb-D-mg＝mamax
Figure BDA0003321642660000071
Thrmaxclimb＝(Thrmaxclimb)ISA×(1-CTc,5×(ΔT-CTc,4))
Figure BDA0003321642660000072
CD＝CD0,CR+CD2,CR×(CL)2
Figure BDA0003321642660000073
wherein, amaxFor maximum aircraft acceleration, ThrmaxclimbMaximum available thrust for the aircraft in climb mode, CTc,1、CTc,2、CTc,3、CTc,4、CTc,5All the parameters are hyper-parameters set in a corresponding aircraft BADA model, have no specific physical significance, are only parameters used for fitting, delta T is an International Standard Atmospheric (ISA) temperature correction value and is related to local actual temperature, HpIs the aircraft barometric altitude, D is the drag experienced by the aircraft, ρ is the air density, v is the aircraft velocity (true airspeed), S is the aircraft wing area, C is the aircraft wing areaDIs the aircraft drag coefficient, CLAs coefficient of lift of the aircraft, CD0,CR、CD2,CRFor a parameter in a specific aircraft BADA model, m is the aircraft mass, g0Taking 9.8m/s as gravity acceleration2. 3) Determining a pitch angle (climbing gradient) interval corresponding to a specific aircraft model and a flight height based on an aircraft performance database:
0≤θsi+wi≤θmax
Figure BDA0003321642660000081
Figure BDA0003321642660000082
Figure BDA0003321642660000083
Thrmaxclimb＝(Thrmaxclimb)ISA×(1-CTc,5×(ΔT-CTc,4))
Figure BDA0003321642660000084
wherein, thetasiFor the time-series climb gradient of the aircraft i, wiPitch angle action available for aircraft i timing, ROCmaxMaximum rate of climb, θ, for aircraftmaxFor maximum climb gradient of the aircraft, M is the aircraft Mach number, T0For ISA atmospheric temperature, 288.15K is taken, f { M } is a piecewise function related to the aircraft speed, delta T is a standard atmospheric temperature correction value, T is a theoretical temperature after the current aircraft barometric altitude is estimated according to the air temperature vertical decreasing rate, betaT,<taking-0.0065K/m as the vertical decreasing rate of the air temperature, wherein R is 287.05m2/(K·s2)，κ＝1.4，
Figure BDA0003321642660000085
Is the derivative of the aircraft vertical velocity with respect to height, m is the aircraft mass, g0Taking 9.8m/s as gravity acceleration2。
4) Determining a pitch angle speed interval corresponding to a specific aircraft model and a flight height based on an aircraft performance database:
wmin≤wi≤wmax
wherein, wmaxFor the maximum head-lifting angular velocity (rad/s), w, of the aircraftminThe maximum low head angular velocity of the aircraft.
Interaction between the two models: the intelligent body selects and executes the optimal action of the aircraft in the current flight state, the environment evaluates the action of the intelligent body according to the set reward function, then the optimal action selection strategy of the intelligent body is updated, the optimal action selection strategy of the aircraft is obtained through continuous interaction of the intelligent body and the environment, and then the optimal climbing flight path is obtained.
As can be seen from fig. 3 and 4, the reward value obtained by the agent through each simulated flight after continuous learning tends to be stable, and the feasibility of solving the four-dimensional flight path planning problem by the reinforcement learning algorithm of the single-step time sequence difference updating optimal strategy is also reflected.
The reinforcement learning algorithm may be embodied as follows:
s1) setting the initial flight state of the aircraft and the position of a target cruising route;
the initial flight state of the aircraft generally selects the height, speed and attitude of the aircraft when the aircraft leaves the runway 15ft away from the ground, and the target cruising waypoint is related to airspace division and flight programming.
S2), after the current aircraft flight state is obtained, generating or updating an available action space in the current aircraft flight state based on the aircraft performance;
s3) calculating the corresponding value (estimated value) of the state-action pair in the agent; wherein, the value is used for measuring the total reward value (global) in the future t time;
s4) making aircraft action selection based on the current flight state of the aircraft;
the intelligent agent decision (action selection) mode specifically comprises the following steps: the roulette wheel determines with a probability P (r ≧ ε) whether the agent takes the action corresponding to the highest expected return for the current state of the aircraft or takes another action.
S5) obtaining the actual state of the aircraft after the selected action is executed-the value (actual value) corresponding to the action.
S6) updating the corresponding value of the state-action pair in the agent according to the actual value:
determining a state s at time ttE S', t time reward rtE.g. R, time t action atAfter leaving the place A, a Markov decision process model is obtained preliminarily, and a function p is defined, namely S 'multiplied by R multiplied by S' multiplied by A → [0, 1-]State transition probability for markov decision process:
p(s'|s,a)＝Pr[st+1＝s',rt+1＝r|st＝s,at＝a],s∈S′,a∈A,s'∈S′
then, the state transition probability:
Figure BDA0003321642660000101
state-action pair desired reward r (s, a):
Figure BDA0003321642660000102
strategy pi (a | s):
π(a|s)＝Pr(at＝a|st＝s)
expected return (predicted value) Gt：
Figure BDA0003321642660000103
Wherein γ ∈ [0,1 ]]For the time-series worth decay parameter, a reward of 1 unit representing the future τ step corresponds to the now-obtained γτPr represents a function of the probability of solution;
action cost function q of Bellman expectation equationπ(s | a) represents:
Figure BDA0003321642660000104
according to the compression mapping principle and Banach fixed-point theorem, there is q*(s | a) compression operator
Figure BDA0003321642660000105
So only the following need to be iterated:
Figure BDA0003321642660000106
the iterative convergence point of the above formula is the fixed point of the Bellman optimal operator, namely the optimal strategy value;
real strategy value U after single step time sequence differencet:t+1 (q)：
Ut:t+1 (q)＝rt+1+γq(st+1,at+1)＝rt+1+γnq(st+n,at+n)
Wherein r ist+1Represents the expected value return of the time sequence of t +1, and q (s, a) represents the reward return brought by taking the action a under the state s of the aircraft;
so the iterative formula has (decrease [ U-q (s, a))]2)：
q(st,at)←q(st,at)+α[gt-q(st,at)]
Wherein, alpha is the learning rate, gtThe real return of environmental feedback when the aircraft makes a decision, including oil consumption cost and time cost, is close to the target reward.
S7) judging whether the aircraft reaches the target cruising route point, if so, finishing the step, otherwise, returning to S2);
s8) judging whether the aircraft reaches the set learning times, if so, ending the step, otherwise, returning to S1);
s9) determining the optimal action of the aircraft in each state according to the iterated optimal state-action-value table.
After the optimal action of each time sequence is obtained by adopting a reinforcement learning algorithm, assuming that the aircraft climbs, and determining the conversion height and the flight distance of the aircraft during the climbing of the steps according to the performance of the aircraft;
the transition altitude of the aircraft may be:
Figure BDA0003321642660000111
Figure BDA0003321642660000112
Figure BDA0003321642660000113
wherein, a0Taking 340m/s and V for ISA atmospheric sound velocityCASCorrecting airspeed for aircraft, HtransFor optimal transfer of altitude, η, for aircrafttransFor the atmospheric temperature ratio at optimum conversion altitude, deltatransTo the atmospheric pressure ratio at the optimum conversion altitude.
As shown in fig. 3, the climb trajectory obtained by the reinforcement learning has a problem that the aircraft operation changes too frequently, and therefore, the climb trajectory needs to be fitted according to the climb mode; wherein, the climbing mode adopts step climbing.
And determining the flat flight distance of the aircraft at the converted altitude according to the characteristics of the air route or the flight program and the control requirement.
According to the converted height and the average flying distance, fitting a climbing track obtained based on the optimal action by adopting a least square method to obtain a four-dimensional climbing track of the aircraft;
and (3) adopting a linear least square method to fit the flight path:
Figure BDA0003321642660000121
f(xi)＝a1xi+a2
wherein J is the sum of the mean square errors of the fitted track point and the original track point, f (-) represents a unary linear regression function, a1、a2As a parameter to be fitted, xiIs the horizontal position of the track sample point, yiIs the track sample point height position.
Based on the same technical scheme, the invention also discloses a software system of the method, and an aircraft four-dimensional climbing track planning system comprises:
an acquisition module: acquiring optional actions in various flight states when an aircraft climbs in a simulated flight experiment; wherein the flight state comprises a horizontal position, an altitude position, a speed and a pitch angle, and the selectable action comprises an acceleration and a pitch angle speed.
A reinforcement learning module: according to the optional actions in the flight state, aiming at the minimum oil consumption and the minimum running time of the cruise waypoint, the reinforced learning algorithm of the single-step time sequence difference updating evaluation strategy is adopted to solve the optimal actions of the aircraft in climbing.
Interactive models in the reinforcement learning algorithm comprise an environment model and an intelligent agent model;
an environment model: calculating the oil consumption and the position change of the aircraft in unit time of operation in the current flight state according to the parameters of the environment where the aircraft is located; calculating an awarding value given by the environment according to the oil consumption and the position change, and taking the awarding value as a parameter for evaluating the current flight state of the aircraft;
the intelligent agent model comprises: an optimal action is selected from the selectable actions.
The reward value calculation formula is as follows:
r(st,at)＝rt 1+rt 2+rt 3
wherein, r(s)t,at) For aircraft in time sequence t, flight state stLower selection action atThe reward value given by the environment;
rt 1obtaining a positive reward value for the aircraft approaching a cruise waypoint;
rt 2obtaining a negative reward value for the aircraft after t time sequence;
rt 3acting on aircraft atFuel is consumed and a negative prize value is achieved.
A track fitting module: and fitting the climbing track obtained based on the optimal action to obtain the four-dimensional climbing track of the aircraft.
The track fitting module comprises:
a conversion height level flight distance determination module: determining the conversion height and the flat flying distance of the aircraft when the stairs climb according to the performance of the aircraft;
a fitting module: and fitting the climbing track obtained based on the optimal action according to the converted height and the plane flight distance to obtain the four-dimensional climbing track of the aircraft.
Based on the same technical solution, the present invention also discloses a computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to execute an aircraft four-dimensional climb trajectory planning method.
Based on the same technical solution, the present invention also discloses a computing device, which includes one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing an aircraft four-dimensional climb trajectory planning.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The present invention is not limited to the above embodiments, and any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are included in the scope of the claims of the present invention which are filed as the application.

Claims (10)

1. A four-dimensional climbing track planning method for an aircraft is characterized by comprising the following steps: the method comprises the following steps:
acquiring optional actions in various flight states when an aircraft climbs in a simulated flight experiment;
according to the optional actions in the flight state, aiming at the minimum oil consumption and the minimum running time of the cruise waypoints, adopting a reinforced learning algorithm of a single-step time sequence difference updating evaluation strategy to solve the optimal actions of the aircraft in climbing;
and fitting the climbing track obtained based on the optimal action to obtain the four-dimensional climbing track of the aircraft.
2. The aircraft four-dimensional climb trajectory planning method according to claim 1, characterized in that: the flight conditions include horizontal position, altitude position, velocity, and pitch angle, and the selectable actions include acceleration and pitch angle velocity.
3. The aircraft four-dimensional climb trajectory planning method according to claim 1, characterized in that: interactive models in the reinforcement learning algorithm comprise an environment model and an intelligent agent model;
an environment model: calculating the oil consumption and the position change of the aircraft in unit time of operation in the current flight state according to the parameters of the environment where the aircraft is located; calculating an awarding value given by the environment according to the oil consumption and the position change, and taking the awarding value as a parameter for evaluating the current flight state of the aircraft;
the intelligent agent model comprises: an optimal action is selected from the selectable actions.
4. The aircraft four-dimensional climb trajectory planning method according to claim 3, characterized in that: the reward value calculation formula is as follows:
r(st,at)＝rt 1+rt 2+rt 3
wherein, r(s)t,at) For aircraft in time sequence t, flight state stLower selection action atThe reward value given by the environment;
rt 1obtaining a positive reward value for the aircraft approaching a cruise waypoint;
rt 2obtaining a negative reward value for the aircraft after t time sequence;
rt 3acting on aircraft atFuel is consumed and a negative prize value is achieved.
5. The aircraft four-dimensional climb trajectory planning method according to claim 1, characterized in that: fitting the climbing track obtained based on the optimal action to obtain the four-dimensional climbing track of the aircraft, wherein the four-dimensional climbing track comprises the following steps:
determining the conversion height and the flat flying distance of the aircraft when the stairs climb according to the performance of the aircraft;
and fitting the climbing track obtained based on the optimal action according to the converted height and the plane flight distance to obtain the four-dimensional climbing track of the aircraft.
6. An aircraft four-dimensional climbing track planning system is characterized in that: the method comprises the following steps:
an acquisition module: acquiring optional actions in various flight states when an aircraft climbs in a simulated flight experiment;
a reinforcement learning module: according to the optional actions in the flight state, aiming at the minimum oil consumption and the minimum running time of the cruise waypoints, adopting a reinforced learning algorithm of a single-step time sequence difference updating evaluation strategy to solve the optimal actions of the aircraft in climbing;
a track fitting module: and fitting the climbing track obtained based on the optimal action to obtain the four-dimensional climbing track of the aircraft.
7. The aircraft four-dimensional climb trajectory planning system of claim 6, wherein: the flight conditions include horizontal position, altitude position, velocity, and pitch angle, and the selectable actions include acceleration and pitch angle velocity.
8. The aircraft four-dimensional climb trajectory planning system of claim 6, wherein: interactive models in the reinforcement learning algorithm comprise an environment model and an intelligent agent model;
an environment model: calculating the oil consumption and the position change of the aircraft in unit time of operation in the current flight state according to the parameters of the environment where the aircraft is located; calculating an awarding value given by the environment according to the oil consumption and the position change, and taking the awarding value as a parameter for evaluating the current flight state of the aircraft;
the intelligent agent model comprises: an optimal action is selected from the selectable actions.
9. The aircraft four-dimensional climb trajectory planning system of claim 8, wherein: the reward value calculation formula is as follows:
r(st,at)＝rt 1+rt 2+rt 3
wherein, r(s)t,at) For aircraft in time sequence t, flight state stLower selection action atThe reward value given by the environment;
rt 1obtaining a positive reward value for the aircraft approaching a cruise waypoint;
rt 2obtaining a negative reward value for the aircraft after t time sequence;
rt 3acting on aircraft atFuel is consumed and a negative prize value is achieved.
10. The aircraft four-dimensional climb trajectory planning system of claim 6, wherein: the track fitting module comprises:
a conversion height level flight distance determination module: determining the conversion height and the flat flying distance of the aircraft when the stairs climb according to the performance of the aircraft;
a fitting module: and fitting the climbing track obtained based on the optimal action according to the converted height and the plane flight distance to obtain the four-dimensional climbing track of the aircraft.
CN202111248553.5A 2021-10-26 2021-10-26 Aircraft four-dimensional climbing track planning method and system Pending CN114115304A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111248553.5A CN114115304A (en) 2021-10-26 2021-10-26 Aircraft four-dimensional climbing track planning method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111248553.5A CN114115304A (en) 2021-10-26 2021-10-26 Aircraft four-dimensional climbing track planning method and system

Publications (1)

Publication Number Publication Date
CN114115304A true CN114115304A (en) 2022-03-01

Family

ID=80376898

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111248553.5A Pending CN114115304A (en) 2021-10-26 2021-10-26 Aircraft four-dimensional climbing track planning method and system

Country Status (1)

Country Link
CN (1) CN114115304A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115206135A (en) * 2022-06-16 2022-10-18 中国电子科技集团公司第二十八研究所 Aircraft instruction height planning method without determining climbing rate

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110673637A (en) * 2019-10-08 2020-01-10 福建工程学院 Unmanned aerial vehicle pseudo path planning method based on deep reinforcement learning
CN112818599A (en) * 2021-01-29 2021-05-18 四川大学 Air control method based on reinforcement learning and four-dimensional track
CN112947541A (en) * 2021-01-15 2021-06-11 南京航空航天大学 Unmanned aerial vehicle intention track prediction method based on deep reinforcement learning
CN113258989A (en) * 2021-05-17 2021-08-13 东南大学 Method for obtaining relay track of unmanned aerial vehicle by using reinforcement learning

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110673637A (en) * 2019-10-08 2020-01-10 福建工程学院 Unmanned aerial vehicle pseudo path planning method based on deep reinforcement learning
CN112947541A (en) * 2021-01-15 2021-06-11 南京航空航天大学 Unmanned aerial vehicle intention track prediction method based on deep reinforcement learning
CN112818599A (en) * 2021-01-29 2021-05-18 四川大学 Air control method based on reinforcement learning and four-dimensional track
CN113258989A (en) * 2021-05-17 2021-08-13 东南大学 Method for obtaining relay track of unmanned aerial vehicle by using reinforcement learning

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115206135A (en) * 2022-06-16 2022-10-18 中国电子科技集团公司第二十八研究所 Aircraft instruction height planning method without determining climbing rate
CN115206135B (en) * 2022-06-16 2024-02-13 中国电子科技集团公司第二十八研究所 Aircraft instruction altitude planning method with uncertain climbing rate

Similar Documents

Publication Publication Date Title
CN110806759B (en) Aircraft route tracking method based on deep reinforcement learning
CA2940044C (en) Flight path optimization using nonlinear programming
CN110727287B (en) Method and system for determining a climbing profile and method of predicting a climbing profile
EP2442199B1 (en) Method and system for vertical navigation using time-of-arrival control
US10041809B2 (en) Aircraft intent processor
CN106557837B (en) Method and device for acquiring continuous descending approach track of airplane
CN107085978A (en) A kind of control aid decision instruction generation method based on required arrival time
CA2856439A1 (en) A flight prediction system
US9196165B2 (en) Assessing feasibility of an aircraft trajectory
CN112818599B (en) Air control method based on reinforcement learning and four-dimensional track
CN101175909A (en) Flight control system having a three control loop design
CN110727286B (en) Method and system for determining a descent profile
CN112180950A (en) Intelligent ship autonomous collision avoidance and path planning method based on reinforcement learning
Porretta et al. Performance evaluation of a novel 4D trajectory prediction model for civil aircraft
CN113268074A (en) Unmanned aerial vehicle flight path planning method based on joint optimization
CN114115304A (en) Aircraft four-dimensional climbing track planning method and system
CN112037583B (en) Optimization guiding method for aircraft altitude layer change program
CN113176787B (en) Power descent trajectory planning online triggering method based on drop point prediction
CN114115314B (en) Sweep angle determining method and system for variant aircraft
Di Vito et al. An overview on systems and algorithms for on-board 3D/4D trajectory management
CN112947073A (en) Model prediction control-based shipboard aircraft sliding track tracking method
Zhang et al. A new Markov decision process based behavioral prediction system for airborne crews
Royer Design of an automatic landing system for the meridian UAV using fuzzy Logic
RU2321524C2 (en) Method and system for control of aircraft at engineless landing
Zhao Trajectory optimization and control for autonomous helicopter shipboard landing

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination