KR102368734B1 - Drone and drone control methods - Google Patents

Abstract

Disclosed are a drone and a drone control method for smoothly enabling autonomous flight of the drone. The present invention uses data collected during the flight process of a drone, thereby capable of allowing the drone to perform autonomous driving smoothly even in a GPS shaded area or in a situation where GPS reception is difficult. The drone includes a GPS receiver, a sensor unit, a storage unit, a GPS coordinate calculation unit, and a control unit.

Description

Drones and drone control methods

The present invention relates to a drone and a drone control method, and more particularly, to a drone and a drone control method that enables autonomous flight to be smoothly performed even in a GPS shaded area of the drone or a situation in which GPS reception is difficult.

A drone is an unmanned aerial vehicle that can fly without a human being. Since around 2010, small drones weighing less than 25 kg have been actively used due to the lightweight and low cost of materials, parts, and equipment, and through this, the demand for drone use in areas that are difficult for humans to reach has rapidly increased.

In the early days of drone research, research on navigation in an environment where GPS (Global Positioning System) was not received was lacking. A lot of research has been done on flight control laws and route planning.

As research advances and sensors such as cameras, LiDAR (Light Detection and Ranging), and IMU (Inertial Measurement Unit) become miniaturized, to overcome the limitations of motion capture systems that can only be used in confined spaces indoors, navigation using the drone’s onboard sensors is improved. Implementation studies have been actively carried out.

Through the resolution of these navigational problems, research on technology development related to the fully autonomous flight of drones for high-speed search, reconnaissance, and detection in rough terrain without human assistance is being actively conducted.

Although the invention disclosed in the prior Korean Patent Application Laid-Open No. 10-2019-0101142 suggests a configuration that allows remote control within the flight set value range, it has a problem that it is difficult to directly apply to autonomous flight, and the GPS reception shadow where GPS reception is difficult When radio interference occurs due to an external device such as a local or drone jammer, there is a problem that autonomous flight is difficult or autonomous flight itself is not possible.

Republic of Korea Patent Publication No. 10-2019-0101142 (published on August 30, 2019)

Therefore, the first object of the present invention to solve this problem is that it is possible to stably autonomously fly even when radio interference and GPS reception are difficult, and it can be received in the process of autonomous flight or general remote control flight. By calculating GPS coordinates inside the drone through deep learning-based learning using drone-related data such as altitude, angular velocity, and acceleration, a drone that enables autonomous flight using the calculated GPS coordinates even if the actual GPS coordinates are not received. will provide

And the second object of the present invention is to enable stable autonomous flight even when radio wave interference and GPS reception are difficult, and to receive altitude, angular velocity, and acceleration in the course of autonomous flight or general remote control flight. It is to provide a drone control method that enables autonomous flight using the calculated GPS coordinates even if the actual GPS coordinates are not received by calculating the GPS coordinates inside the drone through deep learning-based learning using drone-related data such as those.

In order to achieve the first object, the present invention provides a GPS receiving unit for receiving first GPS coordinates, a sensor unit for collecting sensor data, a storage unit for storing changes in the sensor data corresponding to the change of the first GPS coordinates, and the first GPS A GPS coordinate calculator for calculating second GPS coordinates by using the change in the sensor data corresponding to the coordinate change, and a controller for controlling the flight of the drone using the first GPS coordinate and the second GPS coordinate, the sensor data comprising: includes altitude, angular velocity, acceleration, direction information, and distance information of the drone, and the distance information provides a drone including a distance between the drone and the ground or a distance between the drone and a specific object.

The GPS coordinate calculation unit builds a data set using the sensor data that changes according to the movement of the drone, and calculates the second GPS coordinates using the data set with the first GPS coordinates as a target value. Perform a deep learning model. can do.

The control unit compares the second GPS coordinates with the first GPS coordinates, and controls the autonomous flight mode of the drone to operate when the second GPS coordinates are within a preset error range.

In order to achieve the second object, the present invention includes the steps of: receiving the first GPS coordinates by a GPS receiver; collecting sensor data by the sensor; storing changes in the sensor data corresponding to the first GPS coordinates change by the storage , a step of calculating, by a GPS coordinate calculator, a second GPS coordinate using a change in the sensor data corresponding to a change in the first GPS coordinate, and a control unit controlling the flight of the drone using the first GPS coordinate and the second GPS coordinate Including, wherein the sensor data includes altitude, angular velocity, acceleration, direction information, and distance information of the drone, and the distance information includes a distance between the drone and the ground or a distance between the drone and a specific object. provide a way

The step of the GPS coordinate calculator calculating the second GPS coordinates by using the change of the sensor data corresponding to the change of the first GPS coordinates is a data set using the sensor data that the GPS coordinate calculator changes according to the movement of the drone. and performing a deep learning model in which the GPS coordinate calculator calculates second GPS coordinates using the data set using the first GPS coordinates as a target value.

In the step of the controller controlling the flight of the drone using the first GPS coordinates and the second GPS coordinates, the controller compares the second GPS coordinates with the first GPS coordinates, and the second GPS coordinates are within a preset error range. In this case, the method may include controlling the autonomous flight mode of the drone to operate.

The drones and drone control methods described above can stably autonomously fly even when radio wave interference and GPS reception are difficult, and the altitude, angular velocity, and By calculating the GPS coordinates inside the drone through deep learning-based learning using drone-related data such as acceleration, it has the effect of enabling autonomous flight using the calculated GPS coordinates even if the actual GPS coordinates are not received.

1 is a view showing a schematic configuration of a drone according to an embodiment of the present invention.
2 is a diagram showing a schematic configuration of a GPS coordinate calculator, which is one configuration of the present invention.
3 is a diagram showing a schematic flow of a drone control method according to an embodiment of the present invention.
4 is a view for explaining the movement of a drone according to an embodiment of the present invention.

The terms or words used in the present specification and claims are not to be construed as limited in their ordinary or dictionary meanings, and on the principle that the inventor can appropriately define the concept of the term in order to best describe the user's invention. It should be interpreted as meaning and concept consistent with the technical idea of the present invention.

Throughout the specification, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated. In addition, the “… wealth", "… energy", "… However, terms such as “module”, “device” and the like mean a unit that processes at least one function or operation, which may be implemented as a combination of hardware and/or software.

Terms used in the embodiments of the present invention will be briefly described, and the present embodiments will be described in detail.

The terms used in the embodiments of the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, which may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, etc. . In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding embodiments. Therefore, the terms used in the present embodiments should be defined based on the meaning of the term and the overall contents of the present embodiments, rather than the simple name of the term.

In an embodiment of the present invention, terms including an ordinal number such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

Also, in an embodiment of the present invention, the singular expression includes the plural expression unless the context clearly dictates otherwise.

In addition, in the embodiments of the present invention, terms such as "comprises" or "have" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, but one It should be understood that it does not preclude the possibility of the presence or addition of or more other features or numbers, steps, operations, components, parts, or combinations thereof.

In addition, in an embodiment of the present invention, a 'module' or 'unit' performs at least one function or operation, and may be implemented as hardware or software, or a combination of hardware and software. In addition, a plurality of 'modules' or a plurality of 'units' may be integrated into at least one module and implemented by at least one processor, except for 'modules' or 'units' that need to be implemented with specific hardware.

In addition, in an embodiment of the present invention, when a certain part is "connected" with another part, it is not only "directly connected" but also "electrically connected" with another element interposed therebetween. Including cases where there is

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing a schematic configuration of a drone according to an embodiment of the present invention, and FIG. 2 is a diagram showing a schematic configuration of a GPS coordinate calculator, which is a configuration of the present invention.

1 and 2, the drone 100 may include a GPS receiving unit 110, a sensor unit 120, a storage unit 130, a GPS coordinate calculation unit 140 and a control unit 150, The GPS coordinate calculation unit 140 may include a model learning module 141 , a GPS coordinate prediction module 142 , a model storage module 143 , and a model lightweight performance module 144 .

More specifically, the GPS receiver 110 may receive the first GPS coordinates. Here, the first GPS coordinates may be Real-Time Kinematic (RTK) GPS coordinates.

And the sensor unit 120 may collect sensor data.

Here, the sensor data may include altitude, angular velocity, acceleration, direction information, and distance information of the drone, and the distance information may include a distance between the drone and the ground or a distance between the drone and a specific object.

In addition, the sensor data collected by the sensor unit 120 may include information about the driving unit of the drone 100 (output of a motor, the number of motors, movement of the motor, etc.).

To this end, the sensor unit 120 may include an accelerometer, a compass sensor, an inertial measurement unit (IMU), and the like.

Also, the storage unit 130 may store a change in sensor data corresponding to a change in the first GPS coordinates.

More specifically, the storage unit 130 confirms a change in the first GPS coordinates that change as the drone moves, and confirms a change in sensor data corresponding to the change in the first GPS coordinate, of the sensor data corresponding to the change in the first GPS coordinate. Changes can be built into the first database.

Here, the change in the first GPS coordinates may include a change in the first GPS coordinates at the time when the flight ends of the drone from the first GPS coordinates at the time when the flight of the drone starts.

In addition, in the process of building the change of sensor data corresponding to the first GPS coordinate change as the first database, the storage unit 130 stores the hardware specifications (the number of motors, the driver specifications such as output, the shape of the drone, the drone size and weight, etc.) can be reflected to build a second database.

In addition, the GPS coordinate calculation unit 140 may calculate the second GPS coordinates by using a change in sensor data corresponding to a change in the first GPS coordinates.

In addition, the GPS coordinate calculation unit 140 can build a data set using sensor data that changes according to the movement of the drone, and uses the first GPS coordinate as a target value to calculate the second GPS coordinate using the data set. model can be performed.

More specifically, the model learning module 141 may build at least one of the first database and the second database stored by the storage unit 130 as a data set, and use the first GPS coordinates as a target value to set the data set. It is possible to perform deep learning model training to calculate the 2nd GPS coordinates using Here, the deep learning model is not limited to a specific deep learning model.

More specifically, the model learning module 141 may build at least one of the first database and the second database as a data set, and by analyzing the constructed data set, the sensor data corresponding to the first GPS coordinate change Changes can be reversed.

That is, the model learning module 141 can predict the corresponding GPS coordinates based on the change in sensor data, and learn the deep learning model so that the predicted GPS coordinates can reach within a preset similarity range compared to the first GPS coordinates. can be done

Here, to explain in more detail the reverse confirmation of the change in sensor data corresponding to the change of the first GPS coordinates, the altitude and angular velocity of the drone compared to the reference first GPS coordinates based on the first GPS coordinates at the time the drone starts flying By checking the first GPS coordinates after changes in , acceleration, direction information, etc. occur, it will be possible to check how much the first GPS coordinates have changed when a specific change occurs in the reference first GPS coordinates.

Then, when a specific change occurs in the reference first GPS coordinates, it is of course expected that the first GPS coordinates will be derived. Through this, it is possible to predict how the GPS coordinates will change according to changes in sensor data.

Based on this determination, the model learning module 141 analyzes the data set constructed with at least one of the first database and the second database, thereby confirming the change in the sensor data corresponding to the first GPS coordinate change in reverse. , through this, it will be possible to learn the deep learning model to calculate the second GPS coordinates using the data set with the first GPS coordinates as the target value.

In addition, the GPS coordinate prediction module 142 may calculate the second GPS coordinates by performing the deep learning model learned by the model learning module 141 .

In addition, the model storage module 143 may store the deep learning model learned by the model learning module 141 .

In addition, the model lightweight performance module 144 may additionally perform knowledge distillation on the deep learning model stored by the model learning module 141 .

That is, since a large amount of memory, CPU, GPU, etc. are required to calculate the 2nd GPS coordinates for autonomous flight by simply performing a deep learning model, specific coordinates are generated by mapping previously captured image information, images, etc. Thus, technology was developed to perform autonomous flight.

However, since it is essential to map image information and images, a higher level camera than the camera used in the existing drone was required, and since image information and images also require a lot of memory, they could not be used indefinitely, and image information And in order to generate specific coordinates using an image, etc., there is a problem in that it is necessary to check a specific shape and form such as a specific terrain, a feature, and an obstacle.

In order to solve this problem, the present invention applies a change in sensor data corresponding to a change in the first GPS coordinates, not image and image information, to a deep learning model so that the second GPS coordinates can be calculated by targeting the first GPS coordinates.

More specifically, when it is necessary to confirm a specific shape and form, such as a specific terrain, feature, or obstacle, in order to generate specific coordinates using conventional image information and images, autonomous flight was possible in a specific space or indoors. , autonomous flight was not possible when it was impossible to confirm specific terrain and features such as the sea, or specific shapes and forms such as obstacles, or when the terrain change was not significant.

Accordingly, the present invention can construct a data set that responds to changes in the altitude, angular velocity, acceleration, direction information, distance information, etc. of the drone and changes in the first GPS coordinates that occur from the motion of the drone rather than image information and images. By performing a deep learning model that calculates the second GPS coordinates using the data set with the first GPS coordinates as the target, autonomous flight is possible even when it is impossible to confirm specific shapes and forms such as specific terrain, features, and obstacles. Regardless of the drone flight environment, successful autonomous flight can be performed even if radio wave interference occurs due to external devices such as GPS shadow areas and drone jammers.

That is, the model lightweight performance module 144 additionally performs knowledge distillation on the deep learning model stored by the model learning module 141, so that the GPS coordinate prediction module 142 is a lightweight deep learning model. Through this, the heavy algorithm of the existing deep learning model is performed when the drone is charging power on the ground or in standby without flying or autonomous flight, so that the 2nd GPS coordinates through deep learning model execution It is possible to improve the recognition and accuracy of the calculation.

In addition, when additional calculation of the second GPS coordinates is required even when the drone is flying or autonomously flying, the model lightweight performance module 144 performs knowledge distillation for the deep learning model stored by the model learning module 141. By performing additionally, the GPS coordinate prediction module 142 may enable efficient deep learning model execution optimized for the drone's own memory, GPU, CPU, and the like.

And the model lightweight performance module 144 applies automatic machine learning (Automated Machine Learning) to the deep learning model stored by the model learning module 141 to further reduce the weight of the deep learning model, so that the drone internal memory, etc. It can make it possible to perform an optimized deep learning model.

In addition, the model lightweight performance module 144 performs pruning on the deep learning model stored by the model learning module 141, so that the GPS coordinate prediction module 142 performs a pre-stored deep learning model. By removing the network unnecessary for calculating the 2nd GPS coordinates in the process, the network can be reconstructed so that it can be operated even in the embedded computing environment inside the drone.

That is, the model weight reduction performing module 144 performs the deep learning model previously learned by the GPS coordinate prediction module 142, and removes and integrates neurons that do not affect the result in the process of calculating the second GPS coordinates. It is possible to reduce the size of the model while minimizing performance degradation of the learning model.

The controller 150 may control the flight of the drone 100 using the first GPS coordinates and the second GPS coordinates. In addition, the control unit 150 compares the second GPS coordinates with the first GPS coordinates, and as a result of the comparison, when the second GPS coordinates are within a preset error range compared to the first GPS coordinates, the autonomous flight mode of the drone 100 is controlled to operate. can

In addition, the controller 150 may include a flight controller (FC), and in order to control the flight of an unmanned aerial vehicle such as the drone 100 , a receiver of a wireless controller that controls an unmanned aerial vehicle such as a drone and an ESC (Electronic Speed) Controls) can be connected.

3 is a diagram showing a schematic flow of a drone control method according to an embodiment of the present invention.

Referring to FIG. 3 , in the drone control method, the GPS receiver 110 may receive the first GPS coordinates (S330).

And the sensor unit 120 may collect sensor data. (S331)

Here, the sensor data may include altitude, angular velocity, acceleration, direction information, and distance information of the drone 100, and the distance information is the distance between the drone 100 and the ground or the distance between the drone 100 and a specific object. may include

Also, the storage unit 130 may store a change in sensor data corresponding to a change in the first GPS coordinates (S332).

In addition, the GPS coordinates calculation unit 140 may calculate the second GPS coordinates by using a change in sensor data corresponding to the change in the first GPS coordinates (S333).

In addition, the GPS coordinate calculation unit 140 may build a data set using sensor data that changes according to the movement of the drone 100 , and the GPS coordinate calculation unit 140 uses the first GPS coordinate as a target value to set the data set. can be used to perform a deep learning model that calculates the second GPS coordinates.

In addition, the controller 150 may control the flight of the drone 100 using the first GPS coordinates and the second GPS coordinates.

In addition, the control unit 150 compares the second GPS coordinates with the first GPS coordinates, and as a result of the comparison, when the second GPS coordinates are within a preset error range compared to the first GPS coordinates, the autonomous flight mode of the drone 100 is controlled to operate. can

4 is a view for explaining the movement of a drone according to an embodiment of the present invention.

Referring to FIG. 4 , when the drone 100 moves from a point A to a point B, any one of the paths (1), (2) and (3) may be used.

Then, here, the first GPS coordinates of the point A and the first GPS coordinates of the point B will be specified, and in the case of the flight paths of the drone 100 (1), (2) and (3), the changes in the first GPS coordinates, respectively will be different from each other, and sensor data at this time will also be different from each other.

Reflecting this, to explain with reference to FIGS. 1 and 2 , based on the first GPS coordinates at the time point (A) when the drone 100 starts flying, the altitude, angular velocity, acceleration, and direction information of the drone compared to the reference first GPS coordinates By checking the first GPS coordinates after the change, etc., it will be possible to check how much the first GPS coordinates have changed when a specific change occurs in the reference first GPS coordinates.

In this case, (1) changes in the altitude, angular velocity, acceleration, direction information of the drone 100 when moving from A to B along the path and (2) the altitude of the drone 100 when moving from A to B along the path , changes in angular velocity, acceleration, direction information, etc. will be different, and similarly (3) changes in altitude, angular velocity, acceleration, direction information, etc. of the drone 100 when moving from A to B along the path will also be different.

In retrospect, the path of the drone 100 can be predicted through the altitude, angular velocity, acceleration, direction information, etc. of the drone 100 when moving from A to B, and through this, the current GPS of the drone 100 coordinates can be predicted.

By continuously repeating this learning, the GPS coordinate calculator 140 can build a data set using sensor data that changes according to the movement of the drone, and uses the first GPS coordinates as a target value and uses the second GPS coordinates. It will be possible to perform a deep learning model to compute

As described above, the configuration and operation of the drone and the drone control method according to the embodiment of the present invention can be made. Meanwhile, although specific embodiments have been described in the description of the present invention, various modifications are made without departing from the scope of the present invention. can be

Although the present invention has been described with reference to limited embodiments and drawings, the present invention is not limited thereto, and various modifications and variations are possible by those skilled in the art to which the present invention pertains.

A person of ordinary skill in the art related to this embodiment will understand that it may be implemented in a modified form within a range that does not deviate from the essential characteristics of the above description. Therefore, the disclosed methods should be considered in an illustrative and not a restrictive sense. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

Drone: 100 GPS receiver: 110
Sensor unit: 120 Storage unit: 130
GPS coordinate calculator: 140 Model learning module: 141
GPS Coordinate Estimation Module: 142 Model Storage Module: 143
Model lightweight performance module: 144 Control unit: 150

Claims (6)

GPS receiver for receiving the first GPS coordinates;
a sensor unit for collecting sensor data;
Stores the change of the sensor data corresponding to the first GPS coordinate change, the change of the sensor data corresponding to the first GPS coordinate change is built as a first database, and the hardware specification of the drone in the first database is stored a storage unit for constructing a reflected second database;
A GPS coordinate calculation unit for calculating a second GPS coordinate by using a change in the sensor data corresponding to the change in the first GPS coordinate; And
a controller for controlling the flight of the drone using the first GPS coordinates and the second GPS coordinates;
includes,
The GPS coordinate calculation unit
A model for performing deep learning model learning by constructing at least one of the first database and the second database as a data set, and calculating the second GPS coordinates using the data set with the first GPS coordinates as a target value learning module;
a GPS coordinate prediction module for calculating the second GPS coordinates by performing the deep learning model learned by the model learning module;
A model storage module for storing the deep learning model learned by the model learning module; And
a model lightweight performance module for performing knowledge distillation on the deep learning model stored by the model learning module;
including,
The sensor data is
The drone includes altitude, angular velocity, acceleration, direction information, and distance information of the drone, and the distance information includes a distance between the drone and the ground or a distance between the drone and a specific object. According to claim 1,
The GPS coordinate prediction module is
A drone that calculates the second GPS coordinates by performing the lightweight deep learning model while the drone is flying or autonomously flying. According to claim 1,
the control unit
A drone that compares the second GPS coordinates with the first GPS coordinates, and controls the autonomous flight mode of the drone to operate when the second GPS coordinates are within a preset error range. receiving, by a GPS receiver, first GPS coordinates;
collecting sensor data by the sensor unit;
storing, by a storage unit, a change in the sensor data corresponding to a change in the first GPS coordinates;
calculating, by a GPS coordinate calculator, a second GPS coordinate using a change in the sensor data corresponding to a change in the first GPS coordinate; and
controlling, by a controller, the flight of the drone using the first GPS coordinates and the second GPS coordinates;
including,
Storing, by the storage unit, a change in the sensor data corresponding to the change in the first GPS coordinates
constructing, by the storage unit, a change in the sensor data corresponding to the change in the first GPS coordinates as a first database; and
constructing, by the storage unit, a second database reflecting the hardware specifications of the drone in the first database;
including,
The step of the GPS coordinate calculation unit calculating the second GPS coordinates by using the change of the sensor data corresponding to the change of the first GPS coordinates
constructing, by a model learning module, at least one of the first database and the second database as a data set;
performing deep learning model learning such that the model learning module calculates the second GPS coordinates using the data set with the first GPS coordinates as a target value;
calculating, by the GPS coordinate prediction module, the deep learning model learned by the model learning module, the second GPS coordinates;
Storing, by the model storage module, the deep learning model learned by the model learning module; And
performing, by the model lightweight performance module, knowledge distillation on the deep learning model stored by the model learning module;
includes,
The sensor data is
The drone control method includes altitude, angular velocity, acceleration, direction information, and distance information of the drone, and the distance information includes a distance between the drone and the ground or a distance between the drone and a specific object. 5. The method of claim 4,
The step of the model lightweight performing module performing knowledge distillation on the deep learning model stored by the model learning module is
calculating, by the GPS coordinate prediction module, the lightweight deep learning model while the drone is flying or autonomously flying, to calculate the second GPS coordinates;
A drone control method comprising a. 5. The method of claim 4,
The step of the control unit controlling the flight of the drone using the first GPS coordinates and the second GPS coordinates
comparing, by the controller, the second GPS coordinates with the first GPS coordinates, and controlling the autonomous flight mode of the drone to operate when the second GPS coordinates are within a preset error range;
A drone control method comprising a.

Images