KR102559608B1 - Learning method for controlling unmanned aerial vehicle and electronic device for performing the same - Google Patents

Abstract

A technique for learning a control model for controlling an unmanned aerial vehicle by inferring steering information from state information related to a maneuver in which the steering information is missing. A learning method for controlling an unmanned aerial vehicle includes checking state information related to maneuvering of a manned aircraft; estimating steering information for controlling the manned aircraft from state information related to the maneuvering of the manned aircraft; The method may include learning a control model to be mounted on the unmanned aerial vehicle based on the steering information.

Description

Learning method for controlling unmanned aerial vehicle and electronic device performing the same

An embodiment of the present specification relates to a technique for learning a control model to be loaded into an unmanned aerial vehicle by inferring steering information from state information related to maneuvering.

Research related to artificial intelligence pilots controlling unmanned aerial vehicles is being conducted. In order to efficiently train an artificial intelligence pilot, state information (eg, position, speed, angular velocity, etc.) according to aircraft maneuvering of pilots of existing manned aircraft may be used. However, in order to learn the artificial intelligence pilot based on the existing state information, steering information corresponding to the state information is required, but the existing state information is missing the steering information. For example, ACMI (Air Combat Maneuvering Instrumentation), an air combat training system introduced by the Air Force, uses a missile-type Airborne Instrumentation Subsystem (AIS) pod to store fighter maneuvers, but through this, only status information such as the position, attitude, and speed of the fighter is obtained, but the control information entered by the pilot is not obtained. In addition, control information is missing from status information related to civil aircraft operation, off/offline game data such as DCS WORLD, and log data obtained from simulators developed for various purposes. Therefore, there is a need for a technique capable of improving efficiency by learning artificial intelligence pilots by inferring corresponding pilot information from state information related to existing maneuvers.

An embodiment of the present specification is proposed to solve the above problems, and relates to a technique for learning a control model to be mounted on an unmanned aerial vehicle by inferring steering information from state information related to maneuvering. The technical problem to be achieved by the present embodiment is not limited to the technical problems described above, and other technical problems may be inferred from the following embodiments.

In order to achieve the above object, a learning method for controlling an unmanned aerial vehicle according to an embodiment of the present specification includes the steps of checking state information related to the maneuvering of a manned aircraft; estimating steering information for controlling the manned aircraft from state information related to the maneuvering of the manned aircraft; and learning a control model to be mounted on the unmanned aerial vehicle based on the steering information.

According to an embodiment, the state information includes trajectory information, position information, attitude information, speed information, and angular velocity information for the manned aircraft, and the steering information is input to the manned aircraft to output the state information. It may include information input.

According to an embodiment, the step of estimating the steering information may include estimating the steering information at time t by applying inverse dynamics to the state information from time t−n (where n is an integer greater than or equal to 0) to time t+k (where k is a natural number greater than or equal to 1) based on time t.

According to an embodiment, the k at the time t+k may be determined differently according to the type of the manned aircraft.

According to an embodiment, the estimating the steering information may include estimating the steering information at time t using a model learned in advance based on the type of the manned aircraft and the trajectory information.

According to an embodiment, the step of learning the control model may include supervising learning of the control model using the estimated steering information.

According to an embodiment, the supervised learning may include learning the control model with an average value of the plurality of steering information when the estimated steering information is plural.

According to an embodiment, the step of learning the control model may include reinforcement learning of the control model by providing different rewards for each mission based on the estimated steering information.

In order to achieve the above object, a non-transitory computer-readable storage medium according to an embodiment of the present specification includes a medium configured to store computer-readable instructions, wherein the computer-readable instructions when executed by a processor: Checking state information related to maneuvering of a manned aircraft; and estimating steering information for controlling the manned aircraft from state information related to the maneuvering of the manned aircraft. and learning a control model to be mounted on the unmanned aerial vehicle based on the steering information.

In order to achieve the above object, an electronic device according to an embodiment of the present specification includes a memory for storing at least one instruction; and a controller that checks state information related to the maneuvering of the manned aircraft, estimates steering information for controlling the manned aircraft from the state information related to the maneuvering of the manned aircraft, and controls the learning of a control model to be mounted on the unmanned aerial vehicle based on the steering information.

Details of other embodiments are included in the detailed description and drawings.

According to an embodiment of the present specification, one or more of the following effects are provided.

According to an embodiment, an artificial intelligence pilot controlling an unmanned aerial vehicle may be trained using existing state information in which pilot information is omitted. At this time, learning efficiency can be further improved by estimating pilot information corresponding thereto from existing state information and learning artificial intelligence pilots through supervised learning and reinforcement learning. AI pilots who have completed learning can control the unmanned aerial vehicle by inputting pilot information themselves in order to carry out missions.

The effects of the embodiments are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 is a diagram illustrating a method of training an artificial intelligence pilot controlling an unmanned aerial vehicle according to an embodiment.
2 is a diagram illustrating a process of inferring steering information according to an exemplary embodiment.
3 is a diagram illustrating trajectory information of an aircraft over time according to an embodiment.
4 is a diagram for explaining steering information according to an exemplary embodiment.
5 is a flowchart illustrating a learning method for controlling an unmanned aerial vehicle according to an embodiment.
6 is a block diagram of an electronic device according to an exemplary embodiment.

The terms used in the embodiments have been selected as general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but they may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technologies, and the like. In addition, in a specific case, there are also terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the corresponding description. Therefore, terms used in the present disclosure should be defined based on the meaning of the term and the general content of the present disclosure, not simply the name of the term.

In the entire specification, when a part is said to "include" a certain component, it means that it may further include other components, not excluding other components unless otherwise stated. In addition, terms such as “~unit” and “~module” described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

The expression of "at least one of a, b, and c" described throughout the specification may include 'a alone', 'b alone', 'c alone', 'a and b', 'a and c', 'b and c', or 'all of a, b, and c'.

A “terminal” referred to below may be implemented as a computer or portable terminal capable of accessing a learning device or other terminals through a network. Here, the computer includes, for example, a laptop, desktop, or laptop equipped with a web browser, and the portable terminal is, for example, a wireless communication device that ensures portability and mobility, and is a communication-based terminal such as International Mobile Telecommunication (IMT), Code Division Multiple Access (CDMA), W-Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), and all types of handheld-based terminals such as smartphones and tablet PCs. It may include a wireless communication device of.

Hereinafter, with reference to the accompanying drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily carry out the present disclosure. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

1 is a diagram illustrating a method of training an artificial intelligence pilot controlling an unmanned aerial vehicle according to an embodiment.

Referring to FIG. 1 , in order to train an artificial intelligence pilot controlling an unmanned aerial vehicle, not only state information 110 related to maneuvering of a manned aircraft but also pilot information 120 corresponding thereto are required. Here, the steering information 120 is information input by a pilot to control a manned aircraft, such as X-stick, Y-stick, throttle, and rudder. That is, the manned aircraft is controlled based on the steering information 120 input by the pilot, and as a result, the state information 110 related to maneuvering can be sensed.

The existing acquired state information 110 may include only maneuver result information such as the trajectory, position, attitude, speed, and angular velocity of the manned aircraft, and may not include steering information 120 as input information. Therefore, in order to learn the artificial intelligence pilot controlling the unmanned aerial vehicle, the input information, the steering information 120, is required together, and thus the steering information 120 needs to be inferred from the state information 110.

Specifically, the steering information 120 may be inferred by applying a preset estimation method to the state information 110 . For example, adjustment information 120 may be extracted from state information 110 by applying inverse dynamics. At this time, accurate adjustment information 120 may be extracted by applying inverse dynamics in consideration of the type of aircraft (size, power performance, etc.). Alternatively, the steering information 120 for the trajectory may be extracted based on a machine-learned model based on the trajectory included in the state information 110 for each type of aircraft.

The control model 130 is a model learned using the steering information 120 inferred together with the state information 110 related to maneuvering, and may correspond to an artificial intelligence pilot mounted on an unmanned aerial vehicle and controlling the unmanned aerial vehicle.

At this time, the control model 130 may be supervised and learned in consideration of steering information 120 as input information and state information 110 as result information. At this time, when there is a plurality of steering information 120 and state information 110, the control model may be supervised and learned based on the average value of the plurality of steering information 120 and the average value of the plurality of state information 110. For example, in the case of steering information 1 corresponding to state information 1 and steering information 2 corresponding to state information 2 to steering information N corresponding to state information N, the control model can be supervised and learned based on the average value of state information 1 to state information N and the average value of steering information 1 to steering information N.

More specifically, the control model 130 may be reinforced based on different rewards for each mission in addition to supervised learning. Specifically, there are various missions such as take-off and landing related to aircraft control, and important elements may be different for each mission. For example, in the case of a landing mission, factors such as the degree of shaking of the aircraft during landing and the speed limit during landing may be important. Accordingly, different rewards may be allocated according to important factors for each mission, and thus, the control model 130 may perform the corresponding mission better by being reinforced-learned in consideration of the reward for each mission.

According to the embodiment, learning efficiency can be improved by inferring the steering information 120 from the state information 110 related to the maneuver, and supervising and reinforcing the control model 130 by considering the state information 110 and the steering information 120 together. In this case, when only supervised learning is used, the control model 130 is trained to imitate the learning data as it is. However, when reinforcement learning is used together, the learning efficiency of the control model 130 can be further improved according to a reward set for each mission.

2 is a diagram illustrating a process of inferring steering information according to an exemplary embodiment.

Referring to FIG. 2 , state information 210 and state information 220 may correspond to information detected from one trajectory (eg, trajectory, position, attitude, speed, angular velocity, etc.). That is, from one trace information sensed for several times, state information 210 from time t−n (n is an integer greater than or equal to 0) to time t and time t+1 to time t+k (k is a natural number greater than or equal to 1). State information 220 can be identified.

Steering information at time t may be inferred based on state information 210 from time tn to time t and state information 220 from time t+1 to time t+k. That is, based on the state information 210 from the past time tn to the present time t and the state information 220 from the future time t+1 to the time t+k, steering information to be input at the current time t can be inferred. This is because an object with high speed and weight, such as an aircraft, has a trajectory that is not discontinuous but continuously changes, so considering these characteristics, the steering information at time t State information from past time tn to future time t+k may be considered in order to estimate . For example, from trajectory information in the landing process for 0 to 10 seconds, steering information at time 3.7 seconds can be inferred based on state information from time 3.0 seconds to time 3.7 seconds and state information from time 3.8 seconds to time 5 seconds.

At this time, at time t-n and time t+k, n and k may be determined differently in consideration of the type of aircraft. For example, in the case of a relatively heavy aircraft, n and k may be relatively small because change in motion is relatively difficult due to inertia, and in the case of a relatively light aircraft, change in motion is relatively easy due to inertia, so n and k may be relatively large. In addition to the weight of the aircraft, the values of n and k may be determined differently in consideration of information such as maneuvering performance and unit time in which pilot information is input.

3 is a diagram illustrating trajectory information of an aircraft over time according to an embodiment.

Referring to FIG. 3 , it is possible to check the trajectory of an aircraft by time. Steering information at time t, considering state information from time tn to time t (e.g., trajectory, position, attitude, speed, angular velocity, etc.) and state information from time t+1 to time t+k can be inferred.

Specifically, considering the state information from time tn to time t and state information from time t + 1 to time t + k, steering information to be input at time t (eg, X-stick, Y-stick, throttle, rudder, etc.) can be inferred. In addition, steering information to be input at time t+1 can be inferred by considering state information from time t-n+1 to time t+1 and state information from time t+2 to time t+k+1. In addition, steering information to be input at time t+2 can be inferred by considering state information from time t-n+2 to time t+2 and state information from time t+3 to time t+k+2. Similarly, steering information to be input at time t+m can be inferred by considering state information from time t-n+m to time t+m and state information from time t+m+1 to time t+k+m. Based on this method, steering information input for each time in one trajectory of the aircraft can be inferred.

4 is a diagram for explaining steering information according to an exemplary embodiment.

Referring to FIG. 4 , FIG. 410 shows the movement of an aircraft and FIG. 420 shows a device for controlling an aircraft. Figure 420 is just an example, and the device that controls the aircraft may be of a different type. The aircraft can freely move in various directions in the x-axis, y-axis, and z-axis according to the control information input to the control device from the control model that controls the unmanned aerial vehicle. Alternatively, the aircraft may rotate in at least one of a roll (Φ) direction, a pitch (γ) direction, and a yaw (Ψ) direction according to steering information input to a device for controlling the aircraft. That is, the unmanned aerial vehicle may be maneuvered in at least one of the X-axis direction, the Y-axis direction, the Z-axis direction, the roll direction, the pitch direction, and the yaw direction according to the steering information input according to the control model.

5 is a flowchart illustrating a learning method for controlling an unmanned aerial vehicle according to an embodiment.

Referring to FIG. 5 , in step S510 , state information related to the maneuvering of a manned aircraft may be checked. Here, the status information related to maneuvering may include trajectory information, location information, posture information, speed information, and angular velocity information, which are results of the maneuvering of the manned aircraft. At this time, the state information related to the maneuver is the result information of the maneuver and may not include steering information input to the manned aircraft.

In step S520, steering information for controlling the manned aircraft may be estimated from state information related to the maneuvering of the manned aircraft. State information related to maneuvering may not include steering information as result information according to steering information that is input information. That is, state information can be output and sensed by the input of steering information.

In this case, steering information at time t can be estimated by applying inverse dynamics to state information from time tn (n is an integer greater than or equal to 0) to time t+k (k is a natural number greater than or equal to 1) based on time t. This is because an object with high speed and weight, such as an aircraft, has a trajectory that is not discontinuous but continuously changes, so considering these characteristics, the steering information at time t State information from past time tn to future time t+k may be considered in order to estimate .

At this time, at time t−n and time t+k, n and k may be determined differently in consideration of the type of manned aircraft. For example, in the case of a relatively heavy aircraft, n and k may be relatively small because change in motion is relatively difficult due to inertia, and in the case of a relatively light aircraft, change in motion is relatively easy due to inertia, so n and k may be relatively large. In addition to the weight of the aircraft, the values of n and k may be determined differently in consideration of information such as maneuvering performance and unit time in which pilot information is input.

Alternatively, steering information at time t may be estimated using a machine-learned model based on the type and trajectory information of the manned aircraft. More specifically, steering information about a trajectory may be estimated using a model that has been machine-learned in advance. For example, based on the trajectory information 1 to N of aircraft type 1, the trajectory information 1 to M of aircraft type 2, and the trajectory information 1 to Q of aircraft type T, the model can be pre-machine-learned for steering information according to the trajectory information for each aircraft type. At this time, based on the pre-machine-learned model, time-specific steering information corresponding to trajectory information X of aircraft type X may be estimated.

In step S530, a control model to be loaded into the unmanned aerial vehicle may be learned based on the steering information. Based on the pilot information estimated from state information related to the maneuvering of the manned aircraft, a control model that is an artificial intelligence pilot to be loaded into the unmanned aerial vehicle may be learned.

Specifically, a control model may be supervised and learned using the estimated steering information. In this case, when there is a plurality of state information and estimated steering information, the control model may be supervised and learned based on the average value of the plurality of steering information and the average value of the plurality of state information in consideration of the supervised learning algorithm.

More specifically, in addition to supervised learning, reinforcement learning may be performed based on different rewards for each mission. In detail, different rewards may be assigned according to important factors for each mission, and the control model may perform the corresponding mission better by being reinforced-learned in consideration of the reward for each mission.

For reference, reinforcement learning is a type of machine learning, and may be a learning method in which an unmanned aerial vehicle recognizes a current state in an environment and selects an action or action sequence that maximizes a reward among selectable actions. The environment dealt with in reinforcement learning may be given, for example, as a Markov decision process. At any point in time t, the Markov decision process can exist in some state s. The aircraft can take some action a in the corresponding state s, and at the next time point t+1, the Markov decision process can stochastically transition to a new state s', at which time a reward can be obtained. The unmanned aerial vehicle may determine an action or an action sequence at each time step according to a machine-learned model, and thus, a reward may be obtained as it transitions to a new state.

6 is a block diagram of an electronic device according to an exemplary embodiment. 6 shows components related to the present embodiment, but is not limited thereto, and other general-purpose components may be further included in addition to the components shown in FIG. 6 .

The electronic device 600 of FIG. 6 is a device that performs the above-described learning method for controlling the unmanned aerial vehicle, and can be mounted on the unmanned aerial vehicle to control the operation of the unmanned aerial vehicle. Redundant content with the above description may be omitted.

Referring to FIG. 6 , an electronic device 600 may include a memory 610, a controller 620, and a bus (not shown). The memory 610 and the controller 620 may communicate with each other through a bus (not shown). Each of the memory 610 and the control unit 620 may refer to a unit that processes at least one function or operation, and may be implemented as hardware or software, or a combination of hardware and software.

In an embodiment, the electronic device 600 may include a memory 610 that stores various data. For example, at least one instruction for operating the electronic device 600 may be stored in the memory 610 . In this case, the memory 610 and the control unit 620 may perform various operations based on these commands. The controller 620 may perform the aforementioned operations as the command stored in the memory 610 is executed by the controller 620 . Memory 610 may be volatile memory or non-volatile memory. For example, the memory 620 may store data processed by a processor (not shown) and data to be processed. The memory 620 may include random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD-ROM, Blu-ray or other optical disk storage, hard disk drive (HDD), solid state drive (SSD), or flash memory.

In an embodiment, a command is executed by the control unit 620, and the control unit 620 can check state information related to the maneuvering of the manned aircraft, estimate steering information for controlling the manned aircraft from the state information related to the maneuvering of the manned aircraft, and learn and control a control model to be mounted on the unmanned aerial vehicle based on the steering information.

An electronic device or terminal according to the above-described embodiments may include a processor, a memory for storing and executing program data, a permanent storage unit such as a disk drive, a communication port for communicating with an external device, and a user interface device such as a touch panel, a key, and a button. Methods implemented as software modules or algorithms may be stored on a computer-readable recording medium as computer-readable codes or program instructions executable on the processor. Here, computer-readable recording media include magnetic storage media (e.g., read-only memory (ROM), random-access memory (RAM), floppy disk, hard disk, etc.) and optical reading media (e.g., CD-ROM), DVD (Digital Versatile Disc) and the like. A computer-readable recording medium may be distributed among computer systems connected through a network, and computer-readable codes may be stored and executed in a distributed manner. The medium may be readable by a computer, stored in a memory, and executed by a processor.

This embodiment can be presented as functional block structures and various processing steps. These functional blocks may be implemented with any number of hardware or/and software components that perform specific functions. For example, an embodiment may employ integrated circuit configurations such as memory, processing, logic, look-up tables, etc. that may execute various functions by control of one or more microprocessors or other control devices. Similar to components that can be implemented as software programming or software elements, embodiments can be implemented in programming or scripting languages such as C, C++, C#, Python, Java, assembler, etc., including various algorithms implemented as data structures, processes, routines, or combinations of other programming constructs. Functional aspects may be implemented in an algorithm running on one or more processors. In addition, this embodiment may employ conventional techniques for electronic environment setting, signal processing, and/or data processing. Terms such as “mechanism”, “element”, “means” and “composition” may be used broadly and are not limited to mechanical and physical components. The term may include a meaning of a series of software routines in association with a processor or the like.

The foregoing embodiments are merely examples and other embodiments may be implemented within the scope of the claims described below.

Claims (9)

Sensing time-specific status information corresponding to a trajectory according to the maneuvering of the manned aircraft;
estimating steering information for controlling the manned aircraft from the state information for each time period corresponding to the trajectory of the manned aircraft; and
Learning a control model to be mounted on an unmanned aerial vehicle based on the steering information;
The state information includes position information, attitude information, speed information, and angular velocity information for the manned aircraft,
The steering information includes information input to the manned aircraft to output the status information;
The step of estimating the steering information,
Estimating steering information at time t by applying inverse dynamics to the state information from time tn (n is an integer greater than or equal to 0) to time t+k (k is a natural number greater than or equal to 1) based on time t,
A learning method for controlling unmanned aerial vehicles. delete delete According to claim 1,
At the time t + k, the k is determined differently depending on the type of the manned aircraft,
A learning method for controlling unmanned aerial vehicles. According to claim 1,
The step of estimating the steering information,
Estimating steering information at time t using a model learned in advance based on the type of the manned aircraft and the trajectory information,
A learning method for controlling unmanned aerial vehicles. According to claim 1,
The step of learning the control model,
Including the step of supervising and learning the control model using the estimated steering information,
A learning method for controlling unmanned aerial vehicles. According to claim 6,
The step of learning the control model,
Reinforcement learning of the control model by providing different rewards for each mission based on the estimated steering information.
A learning method for controlling unmanned aerial vehicles. As a non-transitory computer-readable storage medium,
a medium configured to store computer readable instructions;
The computer readable instructions, when executed by a processor, cause the processor to:
Sensing time-specific status information corresponding to a trajectory according to the maneuvering of the manned aircraft;
estimating steering information for controlling the manned aircraft from the state information for each time period corresponding to the trajectory of the manned aircraft; and
Learning a control model to be mounted on an unmanned aerial vehicle based on the steering information
including,
The state information includes position information, attitude information, speed information, and angular velocity information for the manned aircraft,
The steering information includes information input to the manned aircraft to output the status information;
The step of estimating the steering information,
Estimating steering information at time t by applying inverse dynamics to the state information from time tn (n is an integer greater than or equal to 0) to time t+k (k is a natural number greater than or equal to 1) based on time t,
A non-transitory computer-readable storage medium that enables a learning method for controlling an unmanned aerial vehicle to be performed. a memory for storing at least one instruction; and
A controller that detects state information for each time period corresponding to the trajectory of the manned aircraft, estimates steering information for controlling the manned aircraft from the state information for each time period corresponding to the trajectory of the manned aircraft, and controls a control model to be installed in the unmanned aerial vehicle based on the steering information to learn;
The state information includes position information, attitude information, speed information, and angular velocity information for the manned aircraft,
The steering information includes information input to the manned aircraft to output the status information;
The control unit applies inverse dynamics to the state information from time tn (n is an integer greater than or equal to 0) to time t+k (k is a natural number greater than or equal to 1) based on time t to estimate steering information at time t,
electronic device.

Images