Disclosure of Invention
It is an object of the present application to provide an unmanned aerial vehicle countermeasure policy decision method and system thereof, which overcome or mitigate at least one technical disadvantage of the known art.
The technical scheme of the application is as follows:
in one aspect, an unmanned aerial vehicle countermeasure policy decision method is provided, including:
abstracting the countermeasure situation of the unmanned aerial vehicle to be decided, and constructing a countermeasure situation abstraction map of the unmanned aerial vehicle to be decided;
identifying the most important unmanned aerial vehicle of the unmanned aerial vehicle to be decided according to the fight situation extraction diagram of the unmanned aerial vehicle to be decided, and constructing a fight relation diagram of the unmanned aerial vehicle to be decided;
and according to the countermeasure relation diagram of the unmanned aerial vehicle to be decided, situation awareness and information interaction of the unmanned aerial vehicle to be decided are carried out, and a countermeasure strategy of the unmanned aerial vehicle to be decided is generated.
According to at least one embodiment of the present application, in the above-described unmanned aerial vehicle countermeasure policy decision method,
the countermeasure situation of the unmanned aerial vehicle to be decided is abstracted, and an countermeasure situation abstraction diagram of the unmanned aerial vehicle to be decided is constructed, specifically:
the method comprises the steps that the body state quantity of the unmanned aerial vehicle to be decided, the state quantity of the unmanned aerial vehicle to be decided carrying the hit object, and the state quantity of the unmanned aerial vehicle to be opposite to the unmanned aerial vehicle carrying the hit object capable of hitting the unmanned aerial vehicle to be decided are used as node embedding quantities of the unmanned aerial vehicle to be decided;
taking the relative state quantity between the unmanned aerial vehicle to be decided and other unmanned aerial vehicles as the edge embedding quantity of the unmanned aerial vehicle to be decided;
and (3) integrating the node embedding quantity and the edge embedding quantity of the unmanned aerial vehicle to be decided, and constructing an fight situation extraction graph of the unmanned aerial vehicle to be decided.
According to at least one embodiment of the present application, in the method for determining an countermeasure policy for an unmanned aerial vehicle, the body state quantity of the unmanned aerial vehicle to be determined includes a vacuum speed, a height, a climbing rate, a three-axis attitude angle, a normal overload, a radar locking signal, an alarm state and a number of carrying hit objects of the unmanned aerial vehicle to be determined;
the state quantity of the hit object carried by the unmanned aerial vehicle to be decided comprises the hit speed, the height, the residual hit time, the distance between the unmanned aerial vehicle to be decided and the hit target unmanned aerial vehicle, the proximity rate between the unmanned aerial vehicle to be decided and the hit target unmanned aerial vehicle, and the entering angle and the beam angle between the unmanned aerial vehicle to be decided and the hit target unmanned aerial vehicle;
the method comprises the steps that the other unmanned aerial vehicle carries a state quantity capable of striking an unmanned object to be decided, wherein the state quantity comprises a striking speed, a height, a residual striking time, a distance between the other unmanned aerial vehicle and the unmanned aerial vehicle to be decided, a proximity rate between the other unmanned aerial vehicle and the unmanned aerial vehicle to be decided, an entering angle between the other unmanned aerial vehicle and the unmanned aerial vehicle to be decided and a beam angle;
the relative state quantity between the unmanned aerial vehicle to be decided and other unmanned aerial vehicles comprises relative distance, proximity rate, altitude difference, entrance angle, beam angle and hitting area information between the unmanned aerial vehicle to be decided and other unmanned aerial vehicles.
According to at least one embodiment of the present application, in the method for determining an unmanned aerial vehicle countermeasure policy, the identifying, according to the to-be-determined unmanned aerial vehicle countermeasure situation extraction graph, the most important unmanned aerial vehicle of the to-be-determined unmanned aerial vehicle, and constructing a to-be-determined unmanned aerial vehicle countermeasure relation graph, specifically includes:
identifying the opposite unmanned aerial vehicle with the greatest threat to the unmanned aerial vehicle to be decided and the own unmanned aerial vehicle with the most cooperative value to the unmanned aerial vehicle to be decided according to the fight situation extraction graph of the unmanned aerial vehicle to be decided;
the method comprises the steps of comprehensively constructing a counter-side unmanned aerial vehicle with the greatest threat to the unmanned aerial vehicle to be decided, a self-side unmanned aerial vehicle with the greatest cooperative value to the unmanned aerial vehicle to be decided, a counter-side unmanned aerial vehicle with the greatest threat to the self-side unmanned aerial vehicle with the greatest cooperative value to the unmanned aerial vehicle to be decided, and constructing a counter-relationship diagram of the unmanned aerial vehicle to be decided by taking the unmanned aerial vehicle to be decided as the self-side unmanned aerial vehicle with the greatest cooperative value.
According to at least one embodiment of the present application, in the method for determining an unmanned aerial vehicle countermeasure policy, according to an abstract drawing of an unmanned aerial vehicle countermeasure situation to be determined, identifying a counterpart unmanned aerial vehicle with the greatest threat to the unmanned aerial vehicle to be determined and a own unmanned aerial vehicle with the most cooperative value to the unmanned aerial vehicle to be determined, specifically:
extracting characteristics of the node embedding quantity of the unmanned aerial vehicle to be decided by using a first neural network;
extracting characteristics of the embedding quantity of the unmanned aerial vehicle to be decided by using a second neural network;
based on the characteristics of the node embedding quantity of the unmanned aerial vehicle to be decided and the characteristics of the inter-edge embedding quantity of the unmanned aerial vehicle to be decided and each other unmanned aerial vehicle, a first graph attention network is used for obtaining high-level characteristic representation of each other unmanned aerial vehicle to be decided;
based on the characteristics of the node embedding quantity of the unmanned aerial vehicle to be decided and the characteristics of the inter-edge embedding quantity of the unmanned aerial vehicle to be decided and each own unmanned aerial vehicle, a second graph attention network is used for obtaining high-level characteristic representation of each own unmanned aerial vehicle to be decided;
and based on the high-level characteristic representation of each other unmanned aerial vehicle to be decided and the high-level characteristic representation of each own unmanned aerial vehicle to be decided, obtaining the other unmanned aerial vehicle with the greatest threat to the unmanned aerial vehicle to be decided and the own unmanned aerial vehicle with the most cooperative value to be decided through softmax operation by a third neural network.
According to at least one embodiment of the present application, in the method for determining a countermeasure policy of an unmanned aerial vehicle, according to a countermeasure relationship diagram of the unmanned aerial vehicle to be determined, situation awareness and information interaction of the unmanned aerial vehicle to be determined are performed, and the countermeasure policy of the unmanned aerial vehicle to be determined is generated, which specifically includes:
based on the characteristics of the node embedding quantity of the unmanned aerial vehicle to be decided, the characteristics of the node embedding quantity of the own unmanned aerial vehicle with the highest cooperative value of the unmanned aerial vehicle to be decided, the characteristics of the node embedding quantity of the other unmanned aerial vehicle with the highest threat of the unmanned aerial vehicle to be decided, the characteristics of the side embedding quantity of the own unmanned aerial vehicle with the highest cooperative value of the unmanned aerial vehicle to be decided, the characteristics of the side embedding quantity of the other unmanned aerial vehicle with the highest threat of the unmanned aerial vehicle with the cooperative value of the unmanned aerial vehicle to be decided, and a third graph attention network is used for obtaining the high-level state perception characteristics of the unmanned aerial vehicle to be decided;
the high-level state potential sensing characteristics of the unmanned aerial vehicle to be decided and the high-level state potential sensing characteristics of the own unmanned aerial vehicle taking the unmanned aerial vehicle to be decided as the unmanned aerial vehicle with the highest cooperative value are interacted, and the high-level state potential sensing fusion characteristics of the unmanned aerial vehicle to be decided are obtained through a fourth neural network;
and generating an countermeasure strategy of the unmanned aerial vehicle to be decided by using a fifth neural network based on the high-level state potential perception fusion characteristic of the unmanned aerial vehicle to be decided.
According to at least one embodiment of the present application, in the method for determining a countermeasure policy of an unmanned aerial vehicle, the countermeasure policy of the unmanned aerial vehicle to be determined includes a maneuvering target unmanned aerial vehicle, maneuvering behaviors, and a hit target unmanned aerial vehicle of the unmanned aerial vehicle to be determined; wherein, the liquid crystal display device comprises a liquid crystal display device,
the maneuvering behaviors of the unmanned aerial vehicle to be decided comprise a horizontal maneuvering direction, a vertical maneuvering direction, a speed and an overload magnitude of the unmanned aerial vehicle to be decided.
According to at least one embodiment of the present application, in the above unmanned aerial vehicle countermeasure policy decision method, the method further includes:
interacting the countermeasure strategy of the unmanned aerial vehicle to be decided with the simulation environment, and collecting a countermeasure strategy sample of the unmanned aerial vehicle to be decided;
training a neural network in the unmanned aerial vehicle countermeasure policy decision-making method by using a countermeasure policy sample of the unmanned aerial vehicle to be decided.
According to at least one embodiment of the present application, in the method for determining a countermeasure policy of an unmanned aerial vehicle, training a neural network in the method for determining a countermeasure policy of an unmanned aerial vehicle by using a countermeasure policy sample of the unmanned aerial vehicle to be determined, specifically:
training the neural network in the unmanned aerial vehicle countermeasure policy decision-making method by using a countermeasure policy sample of the unmanned aerial vehicle to be decided and using a reinforcement learning algorithm based on near-source policy optimization.
Another aspect provides an unmanned aerial vehicle countermeasure policy decision system, comprising:
the to-be-decided unmanned aerial vehicle countermeasure situation abstract diagram construction module is used for abstracting the countermeasure situation of the to-be-decided unmanned aerial vehicle and constructing a to-be-decided unmanned aerial vehicle countermeasure situation abstract diagram;
the to-be-decided unmanned aerial vehicle countermeasure relation diagram construction module is used for identifying the most important unmanned aerial vehicle of the to-be-decided unmanned aerial vehicle according to the to-be-decided unmanned aerial vehicle countermeasure situation extraction diagram and constructing the to-be-decided unmanned aerial vehicle countermeasure relation diagram;
the countermeasures strategy generation module of the unmanned aerial vehicle to be decided is used for carrying out situation awareness and information interaction of the unmanned aerial vehicle to be decided according to the countermeasures relation diagram of the unmanned aerial vehicle to be decided, and generating countermeasures strategies of the unmanned aerial vehicle to be decided;
the system comprises a countermeasure policy sample acquisition module of the unmanned aerial vehicle to be decided, a countermeasure policy analysis module of the unmanned aerial vehicle to be decided, a challenge policy analysis module of the unmanned aerial vehicle to be decided, a;
and the unmanned aerial vehicle countermeasure policy decision neural network training module trains the neural network in the unmanned aerial vehicle countermeasure policy decision method by utilizing a countermeasure policy sample of the unmanned aerial vehicle to be decided.
The application has at least the following beneficial technical effects:
according to the unmanned aerial vehicle countermeasure policy decision-making method, a graph data structure is utilized to abstract the countermeasure situation of the unmanned aerial vehicle to be decided, a countermeasure situation abstract graph of the unmanned aerial vehicle to be decided is built, the most important unmanned aerial vehicle of the unmanned aerial vehicle to be decided is further identified, a countermeasure relation graph of the unmanned aerial vehicle to be decided is built, so that the countermeasure relation of the unmanned aerial vehicle i to be decided is simplified, situation awareness and information interaction of the unmanned aerial vehicle to be decided are carried out based on the countermeasure relation, a countermeasure policy of the unmanned aerial vehicle i to be decided is generated, the processing data amount can be effectively reduced, the efficiency is improved, the difficulty of autonomous countermeasure policy decision-making learning of the unmanned aerial vehicle to be decided can be reduced, and an ideal effect is achieved in large-scale unmanned aerial vehicle countermeasure.
On the other hand, an unmanned aerial vehicle countermeasure policy decision system is provided, which corresponds to the above-disclosed unmanned aerial vehicle countermeasure policy decision method, and the technical effects of the relevant parts of the unmanned aerial vehicle countermeasure policy decision method can be referred to, and are not described herein.
Detailed Description
In order to make the technical solution of the present application and the advantages thereof more apparent, the technical solution of the present application will be more fully described in detail below with reference to the accompanying drawings, it being understood that the specific embodiments described herein are only some of the embodiments of the present application, which are for explanation of the present application, not for limitation of the present application. It should be noted that, for convenience of description, only a portion related to the present application is shown in the drawings, and the rest related portions may refer to a general design, and without conflict, the embodiments and technical features in the embodiments may be combined with each other to obtain new embodiments.
Furthermore, unless defined otherwise, technical or scientific terms used in the description of this application should be given the ordinary meaning as understood by one of ordinary skill in the art to which this application belongs. The terms "upper," "lower," "left," "right," "center," "vertical," "horizontal," "inner," "outer," and the like as used in this description are merely used to indicate relative directions or positional relationships, and do not imply that a device or element must have a particular orientation, be configured and operated in a particular orientation, and that the relative positional relationships may be changed when the absolute position of the object being described is changed, and thus should not be construed as limiting the present application. The terms "first," "second," "third," and the like, as used in the description herein, are used for descriptive purposes only and are not to be construed as indicating or implying any particular importance to the various components. The use of the terms "a," "an," or "the" and similar referents in the description of the invention are not to be construed as limited in number to the precise location of at least one. As used in this description, the terms "comprises," "comprising," or the like are intended to cover an element or article that appears before the term as such, and equivalents thereof, that appears after the term as recited, without excluding other elements or articles.
Furthermore, unless specifically stated and limited otherwise, the terms "mounted," "connected," and the like in the description herein are to be construed broadly and refer to either a fixed connection, a removable connection, or an integral connection, for example; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can also be communicated with the inside of two elements, and the specific meaning of the two elements can be understood by a person skilled in the art according to specific situations.
The present application is described in further detail below with reference to fig. 1-2.
In one aspect, an unmanned aerial vehicle countermeasure policy decision method is provided, including:
abstracting the countermeasure situation of the unmanned aerial vehicle i to be decided, and constructing a countermeasure situation extraction graph GS_i of the unmanned aerial vehicle to be decided;
identifying the most important unmanned aerial vehicle of the unmanned aerial vehicle i to be decided according to the unmanned aerial vehicle countermeasure situation extraction graph GS_i to be decided, and constructing a unmanned aerial vehicle countermeasure relation graph GT_i to be decided;
and carrying out situation awareness and information interaction of the unmanned aerial vehicle i to be decided according to the unmanned aerial vehicle countermeasure relation graph GT_i to be decided, and generating a countermeasure strategy of the unmanned aerial vehicle i to be decided.
For the unmanned aerial vehicle countermeasure policy decision-making method disclosed by the embodiment, it can be understood by those skilled in the art that the situation of the unmanned aerial vehicle to be decided i is abstracted by using the graph data structure, the situation of the unmanned aerial vehicle to be decided is abstracted, the most important unmanned aerial vehicle of the unmanned aerial vehicle to be decided i is further identified, the situation of the unmanned aerial vehicle to be decided is perceived and information is interacted based on the situation of the unmanned aerial vehicle to be decided, the countermeasure policy of the unmanned aerial vehicle to be decided i is generated, the processing data volume can be effectively reduced, the efficiency is improved, the difficulty of autonomous countermeasure policy decision-making learning of the unmanned aerial vehicle to be decided can be reduced, and an ideal effect is achieved in large-scale unmanned aerial vehicle countermeasure.
In some alternative embodiments, the unmanned aerial vehicle countermeasure policy decision method described above,
the countermeasure situation of the unmanned aerial vehicle i to be decided is abstracted, and a countermeasure situation extraction graph GS_i of the unmanned aerial vehicle to be decided is constructed, specifically:
taking the body state quantity Sself_i of the unmanned aerial vehicle i to be decided, the state quantity Smsls_i of the hit object carried by the unmanned aerial vehicle to be decided and the state quantity Smslo_i of the hit object carried by the unmanned aerial vehicle to be decided as the node embedding quantity of the unmanned aerial vehicle i to be decided;
taking the relative state quantity Srel_i between the unmanned aerial vehicle to be decided and the rest unmanned aerial vehicles as the edge embedding quantity of the unmanned aerial vehicle i to be decided;
and (3) integrating the node embedding quantity and the edge embedding quantity of the unmanned aerial vehicle i to be decided to construct an fight situation extraction graph GS_i of the unmanned aerial vehicle i to be decided, wherein the fight situation extraction graph GS_i is a star graph taking the unmanned aerial vehicle i to be decided as a central node, the output degree of the star graph is the number of the rest unmanned aerial vehicles, and the star graph is pointed to the rest airplanes.
In some optional embodiments, in the method for determining a countermeasure policy of an unmanned aerial vehicle, the body state quantity sself_i of the unmanned aerial vehicle i to be determined includes a vacuum speed, a height, a climbing rate, a three-axis attitude angle, a normal overload, a radar locking signal, an alarm state and a number of carrying hits of the unmanned aerial vehicle i to be determined;
the state quantity Smsls_i of the hit object carried by the unmanned aerial vehicle to be decided comprises a hit speed, a height, a remaining hit time, a distance between the unmanned aerial vehicle to be decided and a hit target unmanned aerial vehicle, a proximity rate between the unmanned aerial vehicle to be decided and the hit target unmanned aerial vehicle, and an entering angle and a beam angle between the unmanned aerial vehicle to be decided and the hit target unmanned aerial vehicle;
the method comprises the steps that the opposite unmanned aerial vehicle carries a state quantity Smslo_i capable of striking an unmanned striker to be decided, wherein the state quantity Smslo_i comprises a striking speed, a height, a residual striking time, a distance between the opposite unmanned aerial vehicle and the unmanned aerial vehicle to be decided, a proximity rate between the opposite unmanned aerial vehicle and the unmanned aerial vehicle to be decided, an entering angle between the opposite unmanned aerial vehicle and the unmanned aerial vehicle to be decided and a beam angle;
the relative state quantity Srel_i between the unmanned aerial vehicle to be decided and other unmanned aerial vehicles comprises relative distance, proximity rate, altitude difference, entrance angle, beam angle and hitting area information between the unmanned aerial vehicle to be decided and other unmanned aerial vehicles.
In some optional embodiments, in the method for determining an unmanned aerial vehicle countermeasure policy, the identifying, according to the unmanned aerial vehicle countermeasure situation extraction graph gs_i to be determined, the most important unmanned aerial vehicle of the unmanned aerial vehicle to be determined, and constructing an unmanned aerial vehicle countermeasure relation graph gt_i to be determined specifically includes:
identifying the opposite unmanned aerial vehicle D_i with the greatest threat to the unmanned aerial vehicle to be decided and the own unmanned aerial vehicle J_i with the most cooperative value of the unmanned aerial vehicle to be decided according to the fight situation extraction graph GS_i of the unmanned aerial vehicle to be decided;
the method comprises the steps of comprehensively constructing a to-be-decided unmanned aerial vehicle countermeasure relation graph GT_i, wherein the to-be-decided unmanned aerial vehicle is a central node, the degree of departure is 3, the to-be-decided unmanned aerial vehicle is respectively pointed to a to-be-decided unmanned aerial vehicle D_i with the greatest threat of the to-be-decided unmanned aerial vehicle and a to-be-decided unmanned aerial vehicle J_i with the greatest cooperative value, the to-be-decided unmanned aerial vehicle D_J_i with the greatest threat of the to-be-decided unmanned aerial vehicle and the other unmanned aerial vehicle D_J_i with the greatest cooperative value, and the degree of arrival of the to-be-decided unmanned aerial vehicle is equal to the number of to-be-decided unmanned aerial vehicles J_J_i with the to-be-decided unmanned aerial vehicle as the most cooperative value, and the to-be-decided unmanned aerial vehicles J_j_i with the most cooperative value.
In some optional embodiments, in the method for determining an unmanned aerial vehicle countermeasure policy, according to the to-be-determined unmanned aerial vehicle countermeasure situation extraction graph gs_i, identifying a counterpart unmanned aerial vehicle d_i with the greatest threat to the to-be-determined unmanned aerial vehicle and a own unmanned aerial vehicle j_i with the greatest cooperative value to the to-be-determined unmanned aerial vehicle, specifically:
extracting a characteristic V_i=f_1 (Sself_i, smsls_i, smslo_i) of an inode embedding amount of the unmanned aerial vehicle to be decided by using a first neural network, wherein f_1 ()' is the first neural network;
extracting a characteristic e_ij=f_2 (Srel_ij) of the i-edge embedding quantity of the unmanned aerial vehicle to be decided by using a second neural network, wherein f_2 ()' is the second neural network; e_ij is the characteristic of the embedding quantity of the boundary between the unmanned plane to be decided and the rest jth aircraft; srel_ij is the relative state quantity between the unmanned aerial vehicle to be decided and the rest j-th unmanned aerial vehicles;
based on the feature V_i of the node embedding quantity of the unmanned aerial vehicle to be decided and the feature e_iD of the inter-edge embedding quantity of the unmanned aerial vehicle to be decided and each other unmanned aerial vehicle, a first graph attention network GAT_1 (), a high-level feature representation Ho_iD of each other unmanned aerial vehicle to be decided is obtained;
based on the characteristic V_i of the node embedding quantity of the unmanned aerial vehicle to be decided, the characteristic e_ij of the inter-edge embedding quantity of the unmanned aerial vehicle to be decided and each own unmanned aerial vehicle, obtaining high-level characteristic representation Hf_ij of the unmanned aerial vehicle to be decided by each own unmanned aerial vehicle by using a second graph attention network GAT_2 ();
based on the high-level characteristic representation Ho_iD of each other unmanned aerial vehicle to be decided and the high-level characteristic representation Hf_iJ of each own unmanned aerial vehicle to be decided, a third neural network f_3 (), which is a double-head neural network, is operated by softmax to obtain the other unmanned aerial vehicle D_i with the greatest threat of the unmanned aerial vehicle to be decided and the own unmanned aerial vehicle J_i with the highest cooperative value of the unmanned aerial vehicle to be decided.
In some optional embodiments, in the method for determining an countermeasure policy of an unmanned aerial vehicle, according to the to-be-determined unmanned aerial vehicle countermeasure relationship graph gt_i, situation awareness and information interaction of the to-be-determined unmanned aerial vehicle i are performed, and a countermeasure policy of the to-be-determined unmanned aerial vehicle i is generated, which specifically includes:
based on the characteristic V_i of the node embedded quantity of the unmanned aerial vehicle to be decided, the characteristic V_J_i of the node embedded quantity of the own unmanned aerial vehicle with the highest cooperative value of the unmanned aerial vehicle to be decided, the characteristic V_D_i of the node embedded quantity of the other unmanned aerial vehicle with the highest threat of the unmanned aerial vehicle to be decided, the characteristic V_D_J_i of the node embedded quantity of the other unmanned aerial vehicle with the highest threat of the unmanned aerial vehicle to be decided, the characteristic e_ij_i of the side embedded quantity of the own unmanned aerial vehicle with the highest cooperative value of the unmanned aerial vehicle to be decided, the characteristic e_id_i of the side embedded quantity of the other unmanned aerial vehicle with the greatest threat of the unmanned aerial vehicle to be decided and the side embedded quantity of the unmanned aerial vehicle to be decided, and the third graph attention network GAT_3 (), the high-state perceived characteristic H_i of the unmanned aerial vehicle to be decided is obtained;
the high-level state potential sensing characteristic H_i of the unmanned aerial vehicle i to be decided and the high-level state potential sensing characteristic H_J_J_i of the own unmanned aerial vehicle taking the unmanned aerial vehicle to be decided as the most synergistically valued are interacted, and a fourth neural network f_4 (), so as to obtain the high-level state potential sensing fusion characteristic X_i of the unmanned aerial vehicle i to be decided;
and generating a countermeasure strategy of the unmanned aerial vehicle i to be decided by using a fifth neural network f_5 (), based on the high-level state potential perception fusion characteristic X_i of the unmanned aerial vehicle i to be decided.
In some optional embodiments, in the method for determining a countermeasure policy of an unmanned aerial vehicle, the countermeasure policy of the unmanned aerial vehicle to be determined includes a maneuvering target unmanned aerial vehicle, maneuvering behavior, and a hit target unmanned aerial vehicle of the unmanned aerial vehicle to be determined, and the fifth neural network f_5 () is a three-head neural network, where,
the maneuvering behaviors of the unmanned aerial vehicle to be decided comprise a horizontal maneuvering position, a vertical maneuvering position, a speed and an overload magnitude of the unmanned aerial vehicle to be decided.
In some optional embodiments, the unmanned aerial vehicle countermeasure policy decision method described above further includes:
interacting the countermeasure strategy of the unmanned aerial vehicle i to be decided with the simulation environment, and collecting a countermeasure strategy sample of the unmanned aerial vehicle i to be decided;
training a neural network in the unmanned aerial vehicle countermeasure policy decision-making method by using a countermeasure policy sample of the unmanned aerial vehicle i to be decided.
In some optional embodiments, in the method for determining a countermeasure policy of an unmanned aerial vehicle, training a neural network in the method for determining a countermeasure policy of an unmanned aerial vehicle by using a countermeasure policy sample of the unmanned aerial vehicle i to be determined, specifically:
training the neural network in the unmanned aerial vehicle countermeasure policy decision-making method by using a countermeasure policy sample of the unmanned aerial vehicle i to be decided and using a reinforcement learning algorithm based on near-source end policy optimization.
In some optional embodiments, in the unmanned aerial vehicle countermeasure policy decision method described above, the first neural network f_1 (), the second neural network f_2 (), the third neural network f_3 (), the fourth neural network f_4 (), and the fifth neural network f_5 () are fully connected neural networks.
Another aspect provides an unmanned aerial vehicle countermeasure policy decision system, comprising:
the to-be-decided unmanned aerial vehicle countermeasure situation extraction graph GS_i construction module is used for abstracting the countermeasure situation of the to-be-decided unmanned aerial vehicle i and constructing the to-be-decided unmanned aerial vehicle countermeasure situation extraction graph GS_i;
the to-be-decided unmanned aerial vehicle countermeasure relation graph GT_i construction module is used for identifying the most important unmanned aerial vehicle of the to-be-decided unmanned aerial vehicle i according to the to-be-decided unmanned aerial vehicle countermeasure situation extraction graph GS_i and constructing the to-be-decided unmanned aerial vehicle countermeasure relation graph GT_i;
the countermeasure policy generation module of the unmanned aerial vehicle to be decided is used for carrying out situation awareness and information interaction of the unmanned aerial vehicle to be decided according to the countermeasure relation graph GT_i of the unmanned aerial vehicle to be decided to generate a countermeasure policy of the unmanned aerial vehicle to be decided;
the system comprises a countermeasure policy sample acquisition module of the unmanned aerial vehicle to be decided, a countermeasure policy analysis module and a countermeasure policy analysis module, wherein the countermeasure policy sample acquisition module of the unmanned aerial vehicle to be decided interacts with a simulation environment and acquires a countermeasure policy sample of the unmanned aerial vehicle to be decided;
and the unmanned aerial vehicle countermeasure policy decision neural network training module trains the neural network in the unmanned aerial vehicle countermeasure policy decision method by using a countermeasure policy sample of the unmanned aerial vehicle i to be decided.
For the unmanned aerial vehicle countermeasure policy decision-making system disclosed in the above embodiment, because it corresponds to the unmanned aerial vehicle countermeasure policy decision-making method disclosed in the above embodiment, the description is simpler, the specific relevant points can be referred to the relevant description of the unmanned aerial vehicle countermeasure policy decision-making method part, the technical effects of which can also be referred to the technical effects of the unmanned aerial vehicle countermeasure policy decision-making method relevant part, and the description is omitted here.
Moreover, those skilled in the art should appreciate that the various modules and units of the disclosed apparatus can be implemented in electronic hardware, computer software, or combinations of both, and that the functions are generally described herein in terms of hardware and software, where such functions are implemented in either hardware or software, depending on the specific application and design constraints of the solution, and that one skilled in the art can choose to implement the described functions in a different manner for each specific application and its practical constraints, although such implementation should not be considered beyond the scope of the present application.
In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described and is different from the other embodiments, so that identical and similar parts of each embodiment are mutually referred.
Having thus described the technical aspects of the present application with reference to the preferred embodiments illustrated in the accompanying drawings, it should be understood by those skilled in the art that the scope of the present application is not limited to the specific embodiments, and those skilled in the art may make equivalent changes or substitutions to the relevant technical features without departing from the principles of the present application, and those changes or substitutions will now fall within the scope of the present application.