CN113342031B - Missile track online intelligent planning method - Google Patents
Missile track online intelligent planning method Download PDFInfo
- Publication number
- CN113342031B CN113342031B CN202110538813.6A CN202110538813A CN113342031B CN 113342031 B CN113342031 B CN 113342031B CN 202110538813 A CN202110538813 A CN 202110538813A CN 113342031 B CN113342031 B CN 113342031B
- Authority
- CN
- China
- Prior art keywords
- missile
- network
- algorithm
- planning
- track
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 128
- 238000013439 planning Methods 0.000 title claims abstract description 74
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 74
- 238000012549 training Methods 0.000 claims abstract description 16
- 230000008569 process Effects 0.000 claims description 15
- 238000011156 evaluation Methods 0.000 claims description 14
- 230000004888 barrier function Effects 0.000 claims description 3
- 238000001514 detection method Methods 0.000 claims description 3
- 230000008901 benefit Effects 0.000 abstract description 7
- 230000002787 reinforcement Effects 0.000 abstract description 6
- 230000007547 defect Effects 0.000 abstract description 5
- 238000013473 artificial intelligence Methods 0.000 abstract description 3
- 238000013461 design Methods 0.000 abstract description 2
- 230000000295 complement effect Effects 0.000 abstract 1
- 238000005516 engineering process Methods 0.000 description 5
- 230000006399 behavior Effects 0.000 description 4
- 230000007123 defense Effects 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000002068 genetic effect Effects 0.000 description 2
- 239000011664 nicotinic acid Substances 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000033228 biological regulation Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013440 design planning Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000019637 foraging behavior Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000010355 oscillation Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/107—Simultaneous control of position or course in three dimensions specially adapted for missiles
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Aiming, Guidance, Guns With A Light Source, Armor, Camouflage, And Targets (AREA)
Abstract
The invention discloses a missile track online intelligent planning method, which combines the traditional track planning method APF and a new generation artificial intelligence method A3C represented by deep reinforcement learning, wherein the advantages of the two methods are complementary, so that the defect that the traditional track planning method is easy to fall into local minimum can be overcome, and the problem of low network convergence speed of the artificial intelligence method can be effectively solved. The proposed establishment algorithm specifically comprises the following steps: in the initial stage of terminal guidance, the missile track is planned by using an APF method combining off-line and on-line, and simultaneously, the generated track is used for training an A3C network in a 'dark place' manner to obtain stable network parameters. At this stage, the accuracy of the APF method flight path is far higher than that of the flight path obtained by the A3C algorithm, so that the network training speed can be greatly increased, and the algorithm operation efficiency can be improved. When the A3C network parameters are stable, the algorithm is automatically and quickly switched to A3C from APF through a planned design two-stage quick switching method, and the network stable A3C algorithm is applied to provide the missile suboptimal/optimal planning track.
Description
Technical Field
The invention belongs to the field of aircraft control, and particularly relates to an online intelligent planning method for an aircraft track.
Background
In recent years, missile technology in all countries is continuously developed, so that the reliability, effectiveness, accurate hitting capacity and the like of missiles are greatly improved. In addition, with the wide application and the fusion development of technologies such as artificial intelligence, cloud computing, big data and the like, new technologies are continuously introduced into the military field, so that wars are rapidly developed towards the intelligent direction. As can be seen, the missile performance and the intelligent level in China need to be improved urgently. The missile track planning is one of the key technologies needing to be promoted. The guided missile track planning refers to searching for a feasible and optimal motion track from a starting point to a target point according to a given optimization index under the condition of comprehensively considering the maneuvering performance, the operation environment, the operation task and the like of the guided missile. The good guided missile track planning algorithm can optimize the guided missile flight track, so that the guided missile can actively avoid threat, prevent collision between the guided missiles and reduce the intercepted probability. Therefore, the method for researching the missile track online intelligent planning has practical significance.
The track planning technology has been widely used in systems such as robots and unmanned planes as one of the prerequisites for realizing autonomous control. At present, the main researches on the route planning method at home and abroad are as follows: a method, an Artificial Potential Field (APF), a Random roadmap method (PRM), a Rapid expansion Random Trees (RRT), and various bionic intelligent algorithms.
The method A is a heuristic path searching algorithm, and mainly guides the algorithm to search the direction through a cost estimation function from a starting point to a target point, so as to find the shortest path between the two points. The algorithm has high search efficiency, and can find the optimal path certainly when the heuristic factor meets the monotonicity condition. However, when there are multiple shorter paths, such an algorithm search result is not necessarily optimal, and the search speed may be reduced as the search range is enlarged. In addition, the path planned by the method is relatively close to the obstacle, and the collision risk exists.
The APF method represents the environment as an artificial visual field, wherein a target point generates a gravitational field, the whole space is influenced, and the gravitational values at different positions are different; the obstacle generates a repulsive force field, and the influence range is limited. The object moves in the field, and is considered to move along the gradient descending direction of the resultant force field under the action of the resultant force of the attraction force and the repulsion force. The algorithm has the advantages of simplicity, intuition, high calculation speed, smooth planned flight path and the like. However, when the obstacle environment is complicated, the obstacle tends to fall into a local minimum in the field of view, and the target position cannot be reached.
The PRM method randomly generates a certain number of nodes, connects all the nodes with each other, and deletes the line if the connection line intersects with the obstacle, and finally obtains the obstacle avoidance path between the starting point and the end point. The algorithm has the advantage that the computational complexity of the algorithm only depends on the number of nodes and the complexity of a node connection graph, and is independent of the space size and the dimension. However, the algorithm has significant disadvantages that the operation results of the algorithm are different due to the random generation of the nodes, the quality of the obtained planned path cannot be guaranteed, and even the situation that the path cannot be searched may occur.
The RRT algorithm uses a tree-like growth mode to expand nodes, and random nodes are adopted to guide the tree expansion direction. The method comprises the following specific steps: randomly generating a number, if the number is smaller than a given value, randomly generating a node, otherwise defining a target point as a random node; on the basis, the current node advances by a fixed length to the direction of the random node to obtain a new node, so that a path from the starting point to the target point is obtained. The algorithm has the advantages that the calculation amount of the algorithm is only related to the path expansion step length and is not related to the space dimension, namely the smaller the algorithm step length is, the slower the searching speed is; the larger the step size, the faster the search speed. However, when there are dense obstacles in the environment, the algorithm convergence speed becomes slow; and when the search step size is large, there may be a problem that the path cannot be searched.
The bionic intelligent algorithm mainly comprises a genetic algorithm, an ant colony algorithm, a particle swarm algorithm and the like, and solves the problem of optimization which is difficult to solve by some traditional algorithms by simulating the genetic process or foraging behavior of organisms and the like and utilizing strong optimizing capability. However, such methods have the disadvantages of limited range of problem solutions, slow convergence speed, easy falling into local optimum, and the like.
It can be seen that most of the above methods can only solve the problem of trajectory planning for known static/slow obstacles. With the increasing complexity of combat missions, the missile is required to rapidly and reliably plan the flight path in complex combat environments such as the emergence of a strong maneuvering obstacle target. Deep Reinforcement Learning (DRL) is taken as a booming development representative of machine Learning, strong perception capability of Deep Learning to complex environment is combined with Reinforcement Learning decision-making capability, advantages are complemented, a solution idea is provided for perception decision-making problem of a complex system, and the method is an effective and fastest development method for solving the flight path planning problem at present.
In summary, the proposed flight path planning method has the following disadvantages:
(1) most only solve the problem of path planning for known static/slow obstacles present in the environment;
(2) the real-time performance, reliability and the like of the planned flight path cannot be well guaranteed, so that the planned flight path cannot be directly applied to the missile flight path planning in a complex combat environment.
Disclosure of Invention
Aiming at the problems in the prior art, the invention aims to provide an online intelligent planning method for the missile track, which can carry out real-time, reliable and intelligent planning on the missile track, so that the missile can effectively avoid the attack of the interception missile of an enemy and realize successful defense.
The technical scheme adopted by the invention is as follows:
a missile track online intelligent planning method comprises the following steps:
step 1, adopting an APF method to plan a missile track at a terminal guidance initial stage, training an A3C network, and obtaining A3C stable network parameters, namely obtaining an A3C algorithm with stable network;
step 2, judging the stability of the A3C network; based on the stability of the A3C network, the rapid switching from the APF method to the A3C algorithm is realized to carry out missile track planning;
and 3, planning the missile track by adopting an A3C algorithm with stable network.
Further, in the step 1, an APF method combining off-line and on-line is adopted to plan the missile flight path.
Further, the process of offline APF law trajectory planning is as follows: and establishing a missile target motion model, planning a reference track of the missile by adopting an off-line APF planning algorithm, and replacing a target gravitational field with a gravitational field of the reference track to enable the missile to fly to the reference track.
Further, the process of on-line APF law flight path planning is as follows: considering that the missile launches the intercepting missile for protecting the target in the process of flying the missile to the target by the reference track, when the missile enters the influence range of the intercepting missile, the repulsion field of the intercepting missile needs to be introduced on the basis of the reference track, and the on-line planning is carried out on the obstacle avoidance track of the missile.
Further, the method for implementing the fast switching between the APF method and the A3C algorithm in step 2 includes:
step 2.1, respectively judging the stability of the network parameters aiming at different types of network parameters;
step 2.2, when the A3C network is stable, the route planning algorithm is quickly and automatically switched to the A3C algorithm from an APF method, so that the A3C algorithm with stable network is used to obtain a suboptimal/optimal missile obstacle avoidance track in a complex flight environment;
and 2.3, if the A3C network does not reach stability, continuing to adopt the APF method flight path to train the network.
Further, aiming at the network parameter without the expected value, detecting whether the network parameter is converged, namely detecting whether the network parameter is converged to a certain value, wherein the convergence value is the final training value of the network parameter; if the network parameter converges, it indicates that the network parameter has reached stability. Further, for a network parameter having a desired value, the difference between the network parameter and the desired value is detected, and if the difference is within a given small positive neighborhood, it indicates that the network parameter has stabilized.
Further, the process of planning the missile flight path in the step 3 is as follows:
firstly, acquiring the missile-target distance between a missile and a protected target, the position, the speed and the track angle information of an incoming interception missile, and realizing battle environment detection;
establishing a ground threat degree rapid evaluation method based on the combat environment information to obtain a target threat evaluation value of the incoming intercepted bomb;
converting the obtained target threat assessment value into a pixel value through a color channel, and establishing a target threat situation map reflecting the relative motion relation among the moving body, the target and the barrier;
the method comprises the steps of taking a target threat situation map as an environment state variable of an A3C algorithm, namely as CNN network input of an A3C algorithm, extracting features of a complex combat environment through the CNN network, and updating network parameters of an Actor by adopting a multithreading asynchronous network parameter updating method formed by a single-step Q-Learning method, a single-step Sarsa method, an n-step Q-Learning method and a dominant behavior evaluation method so as to obtain a continuous suboptimal/optimal planning track.
Further, the four methods, namely the single-step Q-Learning method, the single-step Sarsa method, the n-step Q-Learning method and the dominant behavior evaluation method, are respectively used as the methods for each thread of A3C, the four methods are used for exploring the environment in parallel and asynchronously, and the A3C network parameters are updated, so that the correlation among the observed data is reduced, an experience playback pool strategy is avoided, the network training time is reduced, and the stability of the A3C algorithm is improved.
The invention has the beneficial effects that:
compared with most of the flight path planning methods based on deep reinforcement learning, the method for planning the flight path to be researched is APF-A3C, combines the traditional APF method with the A3C method, utilizes APF to plan the initial flight path, uses the obtained flight path to train an A3C network in advance, and adopts the network parameters to basically stabilize the A3C, so that the rapid and reliable planning of the flight path is realized. Therefore, the simulation research algorithm can greatly improve the operation efficiency of A3C and can realize the online intelligent planning of the missile obstacle avoidance track. In addition, compared with most of traditional track planning methods, the method for planning the track by the aid of the APF only in the terminal guidance initial stage and introducing the virtual target point by designing the time-varying reference track gravitational field center effectively avoids the defect that the method is prone to falling into the local minimum. In conclusion, the APF-A3C track planning method is to be established, so that the operation efficiency of the A3C algorithm can be improved, and the defect that the APF algorithm is easy to fall into the local minimum value can be effectively avoided.
Drawings
FIG. 1 is a flowchart of the missile trajectory online intelligent planning algorithm of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, the missile trajectory online intelligent planning method includes the following steps:
step 1, in order to avoid the defect that the algorithm is easy to fall into local optimization when a moving body is close to a target in an APF method, and the route of the missile cannot be planned, and considering that the distance between the missile and the target is far in a final guidance initial stage, the invention plans to plan the route of the missile by adopting an off-line and on-line combined APF method in the stage; and training an A3C network to obtain A3C stable network parameters, namely obtaining the A3C algorithm with stable network. The off-line and on-line missile track planning methods are as follows:
step 1.1, adopting an offline and online combined APF method to plan the missile flight path, and comprising the following specific processes:
step 1.1.1, the course of off-line APF law flight path planning:
firstly, a missile target motion model is established, a reference track of the missile is planned by adopting an off-line APF planning algorithm, and a gravitational field of the reference track is used for replacing a target gravitational field, so that the missile flies to the reference track. In order to prevent the quasi-designed reference track from falling into the local minimum value near the target point, a dynamic gravitational field center is designed, namely the gravitational field center changes along with the change of the flight time and the position of the missile, or the gravitational field center is a function of the flight time of the missile and the distance of the missile, so that the reference track can lose the minimum value characteristic along with the change of the two parameters. The gravitational field is formed by artificially establishing a ground target point to generate a ground attraction gravitational field for the guided missile, and plays a role in attracting the guided missile. The gravitational field center is the target point. In order to avoid that the missile is trapped into local minimum near a target point to cause that the route of the missile reaching the target point cannot be planned, the algorithm establishes a dynamic target point, namely the position of the target point is designed as a function of the flight time of the missile and the distance of the missile, and the position of the dynamic target point is continuously changed along with the increase of the flight time of the missile and the decrease of the distance of the missile, so that the missile escapes from the minimum value.
Step 1.1.2, the process of on-line APF law flight path planning:
considering that an enemy defense system emits an intercepting missile for protecting a target in the process that the missile flies to the target by a reference track, when the missile enters an influence range of the intercepting missile, introducing an intercepting missile repulsive field on the basis of the reference track, and performing online planning on the missile obstacle avoidance track; specifically, the interception missile repulsive force field is a field for artificially establishing a ground barrier to generate repulsive force on a missile, and the repulsive force is generated on the missile. When the obstacle, namely the interception bomb is emitted from the defense base of the enemy, repulsion is generated to the attack bomb. In order to avoid the phenomenon that the online planning flight path falls into local minimum or generates oscillation when the attack bombs, the interception bombs and the targets are collinear, the invention establishes the virtual target. And when the planned flight path falls into the local minimum, the virtual target is used for replacing the actual target to generate a gravitational field, so that the missile quickly leaves the minimum value point and reaches the virtual target point, then the actual target generates the gravitational field, and the planning of the missile flight path is continued. And for a fixed target, planning a flight path of the missile by using an off-line APF method before the missile flies, and binding the planned flight path to a missile-borne computer so that the missile flies to a striking target according to the off-line planned flight path. When the missile flies, the enemy defense system is found to launch the interception bomb and the attack bomb enters the interception bomb influence range through the ground, the air-based radar, the missile loading sensor and the like, and the attack bomb flight path is planned by using an online APF method.
Step 1.2, training an A3C network to obtain A3C stable network parameters, namely obtaining an A3C algorithm with stable network, which is specifically as follows:
since training the A3C network requires a large amount of data, to improve the accuracy of the network, the flight path in both cases must be included. Furthermore, offline and online tracks respectively represent missile flight conditions under the conditions of no obstacle and obstacle, so that the diversity of network training data can be enriched, and the network robustness can be improved by the aid of the data.
Therefore, in order to improve the operating efficiency of the A3C algorithm (Asynchronous adaptive attack-critical), at this stage, a track obtained by an off-line and on-line APF method in the step 1 is used as training data of the A3C algorithm, and the A3C network is trained, so that an A3C network parameter is obtained in advance, and preparation is made for quickly obtaining an effective missile obstacle avoidance track by using the algorithm.
Step 2, based on the two track planning algorithms in step 1, the method for realizing the rapid switching between the APF method and the A3C algorithm comprises the following steps:
in the missile flight process, the stability of the A3C network parameters in the step 1 is detected in real time, and the method specifically comprises the following steps:
step 2.1, the network parameters have a plurality of values, which are classified into an unexpected value type and an expected value type, so that the method for judging the stability of the network parameters aiming at different types of network parameters is as follows:
for the network parameter without the expected value, detecting whether the network parameter is converged, namely detecting whether the network parameter is converged to a certain value, wherein the convergence value is the final training value of the network parameter; if the network parameter converges, it indicates that the network parameter has stabilized.
For a network parameter having an expected value, the difference between the network parameter and the expected value is detected, and if the difference is within a given small positive neighborhood, it indicates that the network parameter has also stabilized. In the present application, the positive decimal value is 0.1.
And 2.2, when the A3C network is stable, the track planning algorithm is quickly and autonomously switched from the APF method to the A3C algorithm, so that the A3C algorithm with stable network is used to obtain the suboptimal/optimal missile obstacle avoidance track in the complex flight environment.
And 2.3, if the A3C network does not reach stability, continuing to adopt the APF method flight path to train the network.
Step 3, adopting a network-stable A3C algorithm to plan the missile track, and the specific process is as follows:
when the algorithm is switched to the A3C algorithm, information such as the missile-eye distance, the position, the speed and the track angle of an incoming intercepting missile between the missile and a protected target is obtained through a missile-borne sensor, a ground-based radar, an air-based radar and the like, and the battle environment detection is realized. And establishing a ground threat degree rapid evaluation method based on the information to obtain a target threat evaluation value of the incoming interception bomb. Specifically, a quick threat level evaluation method can be established by adopting the contents disclosed in the document 1 (Zhaohao, Shiweiwei, Gecanon and the like, the TOPSIS is improved, the multi-time fusion intuition fuzzy threat evaluation is carried out, and the methods are used for control and decision, 2019,34(4): 811-815.).
And converting the obtained target threat assessment value into a pixel value through a color channel, and establishing a target threat situation map reflecting the relative motion relation among the moving body, the target and the obstacle.
The obtained image-form target threat situation map is used as an environment state variable of an A3C algorithm, specifically, CNN network input of an A3C algorithm is used for carrying out feature extraction on a complex combat environment through the CNN network, and an Actor network parameter updating method is adopted to update the Actor network parameters, wherein the multithreading asynchronous network parameter updating method is composed of a single-step Q-Learning method, a single-step Sarsa method, an n-step Q-Learning method and an advantageous behavior evaluation method, so that a continuous-form suboptimal/optimal planning track is obtained. Since the A3C is a multithreading method, the four methods, namely the single-step Q-Learning method, the single-step Sarsa method, the n-step Q-Learning method and the dominant behavior evaluation method, are respectively used as the methods for each thread of the A3C, the four methods are used for exploring the environment in parallel and asynchronously, and the A3C network parameters are updated, so that the correlation among observed data is reduced, an experience playback pool strategy is avoided, the network training time is reduced, and the stability of the A3C algorithm is improved.
On the basis of obtaining the planned flight path, the missile flies along the flight path, namely, the action strategy of the obtained ground acts on the environment to form a new operation environment. Through the process, the complex dynamic environment is continuously sensed, and the guided missile obstacle avoidance track planning under the current environment is realized.
The invention aims to research an APF-A3C algorithm and rapidly plan the route of an incoming interception missile for a missile. The APF method has the advantages of fast resolving, smooth obtained path, easy engineering realization and the like, so the planning flight path can have satisfactory real-time performance by adopting the algorithm. Furthermore, the track planning idea combining off-line and on-line is adopted, so that the track planning reliability of APF regulations is ensured. In addition, the center of a track gravitational field is referred to during the design planning, and a virtual target point is introduced, so that the inherent defects of the APF method, namely the problem of easy falling into a local minimum value, are effectively avoided. Therefore, in the initial stage of terminal guidance, the real-time performance and the reliability of the route planning by adopting the APF rule can be theoretically guaranteed.
Compared with other deep reinforcement learning methods, the method for A3C can acquire more comprehensive environmental information by observing different parts of the environment by adopting various deep reinforcement learning methods, and therefore, the method can acquire a reliably planned flight path. In addition, an APF method flight path is planned to be used for training the A3C network in advance, so that the real-time performance and the reliability of the planned flight path are further guaranteed under the condition of complex combat by the A3C method. From the above, it can be known that the on-line intelligent planning of the missile active evasive flight path is completely feasible by applying the proposed APF-A3C algorithm theoretically.
The above embodiments are only used for illustrating the design idea and features of the present invention, and the purpose of the present invention is to enable those skilled in the art to understand the content of the present invention and implement the present invention accordingly, and the protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes and modifications made in accordance with the principles and concepts disclosed herein are intended to be included within the scope of the present invention.
Claims (7)
1. A missile track online intelligent planning method is characterized by comprising the following steps:
step 1, adopting an APF method to carry out missile track planning at the initial stage of terminal guidance, training an A3C network, and obtaining A3C stable network parameters, namely obtaining an A3C algorithm with stable network;
step 2, judging the stability of the A3C network; based on the stability of the A3C network, the APF method is quickly switched to the A3C algorithm to plan the missile flight path; the method for realizing the rapid switching between the APF method and the A3C algorithm in the step 2 comprises the following steps:
step 2.1, respectively judging the stability of the network parameters aiming at different types of network parameters;
step 2.2, when the A3C network is stable, the route planning algorithm is quickly and automatically switched to the A3C algorithm from an APF method, so that the A3C algorithm with stable network is used to obtain a suboptimal/optimal missile obstacle avoidance track in a complex flight environment;
step 2.3, if the A3C network does not reach stability, continuing to adopt APF method flight path to train the network;
step 3, adopting a network-stable A3C algorithm to plan the missile track;
the process of planning the missile flight path in the step 3 is as follows:
firstly, acquiring the missile-target distance between a missile and a protected target, the position, the speed and the track angle information of an incoming interception missile, and realizing battle environment detection;
establishing a ground threat degree rapid evaluation method based on the combat environment information to obtain a target threat evaluation value of the incoming interception bomb;
converting the obtained target threat assessment value into a pixel value through a color channel, and establishing a target threat situation map reflecting the relative motion relation among the moving body, the target and the barrier;
the method comprises the steps of taking a target threat situation map as an environment state variable of an A3C algorithm, namely as CNN network input of an A3C algorithm, extracting features of a complex combat environment through the CNN network, and updating an Actor network parameter by adopting a multithreading asynchronous network parameter updating method formed by a single-step Q-Learning method, a single-step Sarsa method, an n-step Q-Learning method and an advantageous behavior evaluation method, so as to obtain a continuous suboptimal/optimal planning track.
2. The missile track online intelligent planning method of claim 1, wherein a single-step Q-Learning method, a single-step Sarsa method, an n-step Q-Learning method and a dominant behavior evaluation method are respectively used as the methods for each thread of A3C, the four methods are used for exploring the environment in parallel and asynchronously, and the A3C network parameters are updated, so that the correlation among observation data is reduced, an empirical playback pool strategy is avoided, the network training time is reduced, and the stability of the A3C algorithm is improved.
3. The method for intelligently planning the missile route on line according to claim 1, wherein in the step 1, an APF (active Power Filter) method combining off-line and on-line is adopted for planning the missile route.
4. The missile trajectory on-line intelligent planning method according to claim 3, wherein the off-line APF trajectory planning process comprises the following steps: and establishing a missile target motion model, planning a reference track of the missile by adopting an off-line APF planning algorithm, and replacing a target gravitational field with a gravitational field of the reference track to enable the missile to fly to the reference track.
5. The missile trajectory online intelligent planning method according to claim 3, wherein the online APF flight trajectory planning process comprises the following steps: considering that the missile launches the interception bomb for protecting the target in the process of flying to the target by the reference track, when the missile enters the influence range of the interception bomb, the repulsion field of the interception bomb needs to be introduced on the basis of the reference track, and the on-line planning is carried out on the obstacle avoidance track of the missile.
6. The missile trajectory on-line intelligent planning method according to claim 1, wherein for a network parameter without an expected value, whether the network parameter is converged is detected, that is, whether the network parameter is converged to a certain value is detected, and the converged value is a final training value of the network parameter; if the network parameter converges, it indicates that the network parameter has reached stability.
7. A missile trajectory on-line intelligent planning method in accordance with claim 1 wherein for a network parameter having an expected value, the difference between the network parameter and the expected value is detected, and the difference is within a given small positive neighborhood, if so, it indicates that the network parameter has stabilized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110538813.6A CN113342031B (en) | 2021-05-18 | 2021-05-18 | Missile track online intelligent planning method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110538813.6A CN113342031B (en) | 2021-05-18 | 2021-05-18 | Missile track online intelligent planning method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113342031A CN113342031A (en) | 2021-09-03 |
| CN113342031B true CN113342031B (en) | 2022-07-22 |
Family
ID=77469045
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110538813.6A Active CN113342031B (en) | 2021-05-18 | 2021-05-18 | Missile track online intelligent planning method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113342031B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107219857A (en) * | 2017-03-23 | 2017-09-29 | 南京航空航天大学 | A kind of unmanned plane formation path planning algorithm based on three-dimensional global artificial potential function |
| CN108387232A (en) * | 2018-01-30 | 2018-08-10 | 河北科技大学 | The flying object path planning method of evolution algorithm based on Artificial Potential Field |
| CN110956662A (en) * | 2019-11-29 | 2020-04-03 | 北京邮电大学 | Carrier obstacle avoidance method and device and electronic equipment |
| CN112666939A (en) * | 2020-12-09 | 2021-04-16 | 深圳先进技术研究院 | Robot path planning algorithm based on deep reinforcement learning |
-
2021
- 2021-05-18 CN CN202110538813.6A patent/CN113342031B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107219857A (en) * | 2017-03-23 | 2017-09-29 | 南京航空航天大学 | A kind of unmanned plane formation path planning algorithm based on three-dimensional global artificial potential function |
| CN108387232A (en) * | 2018-01-30 | 2018-08-10 | 河北科技大学 | The flying object path planning method of evolution algorithm based on Artificial Potential Field |
| CN110956662A (en) * | 2019-11-29 | 2020-04-03 | 北京邮电大学 | Carrier obstacle avoidance method and device and electronic equipment |
| CN112666939A (en) * | 2020-12-09 | 2021-04-16 | 深圳先进技术研究院 | Robot path planning algorithm based on deep reinforcement learning |
Non-Patent Citations (1)
| Title |
|---|
| 基于深度时空Q网络的定向导航自动驾驶运动规划;胡学敏等;《计算机应用》;20200710(第07期);第1919页-第1925页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113342031A (en) | 2021-09-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108153328B (en) | Multi-missile collaborative track planning method based on segmented Bezier curve | |
| CN113741525B (en) | Policy set-based MADDPG multi-unmanned aerial vehicle cooperative attack and defense countermeasure method | |
| EP1974305A2 (en) | Uav decision and control system | |
| Xia et al. | Multi—UAV path planning based on improved neural network | |
| CN112558634B (en) | Multi-missile cooperative guidance method and system | |
| CN117035435A (en) | Multi-unmanned aerial vehicle task allocation and track planning optimization method in dynamic environment | |
| Chipade et al. | Herding an adversarial swarm in an obstacle environment | |
| Li et al. | Intelligent path planning algorithm for UAV group based on machine learning | |
| Yan et al. | Research on uav path planning based on improved artificial potential field method | |
| Zhang et al. | Multi-UAV cooperative Route planning based on decision variables and improved genetic algorithm | |
| CN113342031B (en) | Missile track online intelligent planning method | |
| Yan et al. | A Path Planning Algorithm based on Artificial Potential Field Method and Ant Colony Algorithm | |
| Wang et al. | MPC and TGFC for UAV real-time route planning | |
| Yin et al. | Research on the operational mode of manned/unmanned collaboratively detecting drone swarm | |
| Wu et al. | A distributed multi-robot cooperative hunting algorithm based on limit-cycle | |
| Liu et al. | Multi-UAV collaborative search and strike based on reinforcement learning | |
| CN115686075A (en) | Unmanned aerial vehicle cluster multi-target collaborative search algorithm based on combination of particle swarm and artificial potential field | |
| Chen et al. | The Pursuit-Evasion Game Strategy of High-Speed Aircraft Based on Monte-Carlo Deep Reinforcement Learning | |
| Gao et al. | Roadmap-restricted multi-robot collaborative hunting method based on improved artificial potential field | |
| Huang et al. | Multi-UCAV cooperative autonomous attack path planning method under uncertain environment | |
| Guopeng et al. | Research on Path planning of Three-Dimensional UAV Based on Levy Flight Strategy and Improved Particle Swarm Optimization Algorithm | |
| Fu | Path planning of ucav based on a modified geesepso algorithm | |
| Zhao et al. | Reinforcement learning based spacecraft autonomous evasive maneuvers method against multi-interceptors | |
| CN114077263A (en) | Unmanned aerial vehicle intermediate-distance air combat tail biting path planning method | |
| Chen | Research on Path Planning of Soccer Robot Based on Potential Field Improved Ant Colony Algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |