KR102467859B1 - A method for performing automatic control using an unmanned aerial vehicle capable of autonomous flight and a ground control system therefor - Google Patents

Abstract

The present invention relates to a method for performing autonomous pest control using an unmanned aerial vehicle capable of autonomous flight and a ground control system therefor The method comprises, according to an embodiment of the present invention, comprises the steps of: displaying a user interface by executing a ground control program for autonomous flight and autonomous pest control of an unmanned aerial vehicle executed on a computing device; displaying a map image matching the received address information and the information on a plurality of work areas and works overlapping the map image; displaying the information on pest control status, flight path, and the state of the unmanned aerial vehicle about a selected work area among the plurality of work areas; and displaying at least one of image information of the selected work area filmed by the unmanned aerial vehicle while flying, information on a path along which the unmanned aerial vehicle has already flown, and information on the current location of the unmanned aerial vehicle within the selected work area.

Description

A method for performing automatic control using an unmanned aerial vehicle capable of autonomous flight and a ground control system therefor}

The present invention relates to a method for performing autonomous control using an unmanned aerial vehicle capable of autonomous flight and a ground control system therefor.

Currently, domestic agriculture is using unmanned aerial vehicles to replace the aging of the workforce for the application of the 4th industry. The importance of real-time information analysis has increased through agricultural unmanned aerial vehicle control, aerial surveillance, image analysis, etc., and is rapidly increasing in line with the demand of the unmanned aerial vehicle industry. Recently, agriculture in the 4th industrial revolution has changed to smart agriculture to resolve aging and reduce production costs, and ICT (Information and Communications Technologies) technology has been grafted to increase efficiency of existing methods and optimize profits to achieve smart agriculture through continuous management. Agricultural development is unfolding all over the world. A smart agricultural system using unmanned aerial vehicles is rapidly emerging as a role to solve problems caused by the aging of agriculture by supporting farmers' control work and increasing efficiency. However, in terms of operation, the efficiency of unmanned aerial vehicles is low, and the necessity is decreasing due to the difference in perception between operators and consumers regarding the utilization of unmanned aerial vehicles. In particular, there was no accurate standard for operating standards and spraying effects, which acted as a part in which reliability decreased from the consumer's point of view. In the end, research on the analysis of the utilization of unmanned aerial vehicles is rare, and there is a large deviation in control efficiency depending on the piloting ability of unmanned aerial vehicles. Therefore, the development of an unmanned aerial vehicle capable of performing pest control while autonomously driving has been required, and autonomous driving of an unmanned aerial vehicle is being applied using various technologies developed to date. In particular, there are many attempts to utilize self-driving unmanned aerial vehicle technology in pest control that requires a lot of labor and uniform work processing of tens of thousands of square meters. In addition, the need for an integrated control system for the management of the flight status, control status, and control map of the self-driving unmanned aerial vehicle is also emerging.

Korean Patent Publication No. 10-2020-0072215

The present invention enables control management plans to be managed in an integrated manner, thereby constructing and analyzing big data to increase the yield of crops through mid- to long-term work plans. A method and a ground control system for the same are provided.

In addition, the present invention is possible to replace the existing manpower with control using an unmanned aerial vehicle, and enables the settlement of the control operation system by increasing the reliability of the consumer.

In addition, the present invention solves the non-uniformity of drug spraying through the manual operation of the unmanned aerial vehicle control operator and enables the construction of a basic operating system capable of identifying the characteristics of the unmanned aerial vehicle through a smart automatic mode.

The embodiment includes the steps of displaying a user interface by executing a ground control program for autonomous flight and autonomous control of an unmanned aerial vehicle executed on a computing device; displaying a map image matching the received address information and a plurality of work areas and work information overlapping the map image; and displaying information on the control status, flight path, and state of the unmanned aerial vehicle of the selected working area among the plurality of working areas, including image information obtained by photographing the selected working area while the unmanned aerial vehicle is flying, and flight of the unmanned aerial vehicle. It is possible to provide a method of performing autonomous control using an unmanned aerial vehicle capable of autonomous flight displaying at least one of path information already flown along a route and current location information of the unmanned aerial vehicle within the selected work area.

In another aspect, the user interface may include displaying a cadastral map image matching a searched address in response to selection of a cadastral map affordance; and displaying an image taken by an unmanned aerial vehicle of the selected point in response to the selection of a point on the displayed cadastral map image on the selected point; How to do it can be provided.

In another aspect, the user interface displays a cadastral map image matching the searched address in response to selection of the cadastral map affordance, and overlaps and displays an image of a region on the cadastral map image on which the unmanned aerial vehicle is displayed, overlapping the cadastral map image; and in response to selection of an area where the captured image and the cadastral map image do not overlap, loading an image of a selected point and matching the captured area to the non-overlapping area; Thus, it is possible to provide a method for performing autonomous control.

In another aspect, displaying a route setting interface displaying a map image in response to selection of a route setting affordance on the user interface; and setting a flight path connecting the selected points in response to the selection of a plurality of points on the map image displayed on the path setting interface, and reproducing the captured image while the UAV moves along the set flight path. In addition, it is possible to provide a method of performing autonomous control using an unmanned aerial vehicle capable of autonomous flight in which an image playback speed is changed based on the flight speed of the unmanned aerial vehicle set on the route setting interface.

In another aspect, displaying type information of a photographed subject on a map image displayed on the route setting interface; displaying a grid map in an area matching the selected item in the map image displayed on the route setting interface in response to selection of any one of a plurality of items in the type information; displaying a grid map for editing allowing the flight path of the unmanned aerial vehicle to be set and modified within the grid map; and providing an interface allowing setting and modifying detailed flight paths in the selected partial grid in response to selection of one of the plurality of partial grids, wherein the grid map is composed of a plurality of partial grids. It is possible to provide a method for performing autonomous control using an unmanned aerial vehicle capable of autonomous flight.

In another aspect, displaying a route setting interface displaying a map image in response to selection of a route setting affordance on the user interface; and setting a plurality of layers through which the unmanned aerial vehicle can fly in response to a selection of a flight layer setting affordance, in the path setting interface, wherein the setting of the plurality of layers comprises the subject from the surface scanning data. Obtaining a point cloud of , identifying the subjects based on the analysis of the point cloud, and setting a plurality of layers connecting specific points of the subjects based on the analysis of the point cloud, thereby selecting one of the plurality of layers. On the layer of the UAV, set a flight path that enables autonomous navigation, and when the UAV recognizes an obstacle on the flight path, move to avoid and move to a layer different from the layer assigned to the UAV. Maintaining the shape of the flight path on the layer and providing an obstacle avoidance simulation interface that simulates flying on the other layer; may be

The embodiment provides a method for performing autonomous control using an unmanned aerial vehicle capable of autonomous flight that provides real-time location information and control status information of an unmanned aerial vehicle during control activities and implements an automatic control function, and provides a ground control system therefor. can do.

The embodiment may provide a method for enabling easy setting of a work area, a work type, a flight type, and a flight route for work on various map images.

Embodiments may provide a method and system for enabling unmanned aerial vehicle image upload and cadastral map editing for correcting inconsistent parcels between the cadastral map and current status.

1 is a schematic diagram of a ground control system system according to an embodiment of the present invention.
2 to 13 schematically show a user interface of a ground control program for autonomous flight and autonomous control of an unmanned aerial vehicle installed and executed in a computing device.

Since the present invention can apply various transformations and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention, and methods for achieving them will become clear with reference to the embodiments described later in detail together with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms. In the following embodiments, terms such as first and second are used for the purpose of distinguishing one component from another component without limiting meaning. Also, expressions in the singular number include plural expressions unless the context clearly dictates otherwise. In addition, terms such as include or have mean that features or elements described in the specification exist, and do not preclude the possibility that one or more other features or elements may be added. In addition, in the drawings, the size of components may be exaggerated or reduced for convenience of explanation. For example, since the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of description, the present invention is not necessarily limited to the illustrated bar.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when describing with reference to the drawings, the same or corresponding components are assigned the same reference numerals, and overlapping descriptions thereof will be omitted. .

1 is a schematic diagram of a ground control system system according to an embodiment of the present invention.

Referring to FIG. 1 , an unmanned aerial vehicle 30 may autonomously fly along a flight area 11 based on a generated autonomous navigation map. Here, the flight area 11 may be defined as an area in which at least one of various preset tasks is performed while the unmanned aerial vehicle 30 flies.

The unmanned aerial vehicle 30 capable of autonomous navigation may be a multi- or single-rotor unmanned aerial vehicle (UAV) or a hybrid airship. The unmanned aerial vehicle 30 may include a gyroscope, an accelerometer, a magnetometer (compass), a barometer, a sonar, an optical flow sensor, an energy management module, a voltage management module, and a GPS module. Various sensors can provide motor input, altitude, pitch, roll, direction, position, attitude, high-precision absolute and relative position, obstacle detection, distance detection, speed control and digital wind speed detection. In some embodiments, the flight speed of the unmanned aerial vehicle 30 according to a change in load weight may be controlled while spraying fertilizer on crops or performing other field work.

The unmanned aerial vehicle 30 may include a frame, a motor mounted on the frame, and electronic and communication equipment attached to the frame. The number of motors can be three (tri-copter), four (quad-copter), six (hexa-copter), eight (octo-copter) or more depending on the thrust required.

The computing device 200 may include a portable terminal 200a and/or a desktop or server 200b. The computing device 200 may control driving of the unmanned aerial vehicle 30 and generate an autonomous navigation map.

The terminal 200a may receive selection of various affordances displayed. Selection here may be a touch input by way of example. In addition, the terminal 200a may determine selection, movement of the selection, and whether to end the selection, and the movement of the selection may exemplarily be a continuous touch input and movement of the touch input position.

The desktop 200b may receive selection of various affordances displayed. Selection here may be an input of a command signal by an external input device such as a mouse. In addition, the desktop 200b can determine selection, movement of the selection, and whether to end the selection. The movement of the selection is, for example, a continuous input of a command signal by an external input device and movement of the position of the input of the command signal. It can be, but is not limited thereto.

2 to 13 schematically show a user interface of a ground control program for autonomous flight and autonomous control of an unmanned aerial vehicle installed and executed in a computing device.

After the ground control program 100 storable in a recording medium is executed on the computing device 200, user information is entered and the computer program 100 goes through a log-in procedure to provide a user interface as exemplarily shown in FIG. 2 . (110) may be displayed.

The user interface 110 may display the main menu affordance 112 . The main menu affordance 112 may include map affordance, route setting affordance, status information affordance, control status affordance, and cadastral map affordance.

The computing device 200 may provide various user interfaces in response to selection of any one of a plurality of affordances in the main menu affordance 112 .

The user interface 110 may display the first map image im1 matching address information input to the search window 111 .

The first map image im1 may be an aerial photographic image, but is not limited thereto.

On the first map image im1, the plurality of work areas 113 may be overlapped and displayed on the first map image im1. The plurality of work areas 113 may be displayed as dotted lines drawn along the boundaries of the work areas to be distinguished from each other, but are not limited thereto.

The plurality of work areas 113 may be displayed separately from each other according to work conditions. Illustratively, the work area 113a where the work is completed and the work area 113b where the work is in progress may be displayed separately from each other.

In some embodiments, among the plurality of work areas 113, they may be grouped according to common characteristics such as a control method, a control operator, and land owner information, and may be displayed in a form in which groups can be distinguished.

The task summary information 114 corresponding to each of the plurality of work areas 113 may be overlapped with the first map image im1 and displayed. The work summary information 114 may include summary information of various pieces of information such as location information of the work area and control information.

Referring to FIG. 3 , the computing device 200 provides detailed work area information 110a of the selected work area 113 in response to selection of any one of the plurality of work areas 113 displayed on the first map image im1. ) can be displayed.

The work area detailed information 110a displayed on the user interface 110 includes the second map image im2 and the third map image im3, flight path information 115 and state information 116 of the unmanned aerial vehicle 30. can include

The second and third map images im2 and im3 may be aerial photographic images, but are not limited thereto.

The second map image im2 may be an enlarged image of the selected work area 113 . In addition, information on the station si closest to the selected work area 113 may be displayed on the third map image im3. The unmanned aerial vehicle 30 may perform charging of batteries, installation or exchange of tool sets necessary for work, charging of chemicals such as fertilizers or pesticides to be sprayed on the work area, etc. at the station (si), but is not limited thereto. not.

The flight path information 115 may be information about a path on which the unmanned aerial vehicle 30 set based on the set autonomous navigation map flies on the selected work area 113 . Exemplarily, a boundary line corresponding to the work area 113 is shown, and a flight path is displayed within the boundary line, so that the flight path of the unmanned aerial vehicle 30 on the selected work area 113 can be checked.

The state information 116 may include information about the total amount of the battery of the unmanned aerial vehicle 30, the total amount of stored chemicals, their usage, flight time information, and the current state of control. Information included in the status information 116 may vary depending on the completion of work, during work, and before work of the work area 113 .

Referring to FIG. 4 , the computing device 200 may display status information of a task matched with a project input in the search window 111 in response to selection of a status information affordance in the main menu affordance 112 .

For example, when Project 1 is searched for, status information of tasks related to Project 1 may be displayed.

The user interface 110 may display an image captured by the unmanned aerial vehicle 30 working in the work area as a real-time image im4.

The computing device 200 may display the enlarged or reduced real-time image im4 in response to selection of an enlargement affordance (+ sign) or a reduction affordance (-sign) of the real-time image im4. The computing device 200 transmits a signal commanding zoom-in or zoom-out of the camera mounted on the unmanned aerial vehicle 30 in response to selection of an enlargement affordance (+ sign) or a reduction affordance (-sign) of the real-time image im4. (30) can be provided. The unmanned aerial vehicle 30 may adjust the focus of a camera in response to receiving a signal for a zoom-in or zoom-out command, and transmit the photographed image to the computing device 200 in real time.

In addition, the user interface 110 may display pest control progress information 117 including information on the progress of pest control, the amount of battery charge of the unmanned aerial vehicle 30, and the remaining amount of fertilizer. Control progress status information 117 may include information related to various other control progress.

In addition, the user interface 110 may display real-time control route information (118). The real-time control status information 118 may display a work area 113, a flight path of the unmanned aerial vehicle 30, and a path already flown while the unmanned aerial vehicle 30 is working.

An already flown path here may be displayed overlapping with at least a part of the flight path.

In addition, the user interface 110 may display real-time location information 119 of the current unmanned aerial vehicle 30 within the work area 113 .

Referring to FIG. 5 , the computing device 200 may display registered control status information in response to selection of the control status affordance in the main menu affordance 112 .

The user interface 110 may display information about the registered work area, the location of the work area, the work status, the person in charge, and the work progress in relation to the control status information.

In addition, among the displayed control status information, the user interface 110 may display only information matching information input to the search window.

In addition, the user interface 110 may display status information related to the selected item as shown in FIG. 4 in response to selection of the detail view affordance.

Referring to FIG. 6 , the computing device 200 may display a cadastral map image im5 of a region matching address information input in the search window 111 in response to selection of the cadastral map affordance in the main menu affordance 112. have.

The computing device 200 may overlap and display a video image of the unmanned aerial vehicle 30 photographing a position on the cadastral map corresponding to the selected point on the displayed cadastral map image im5 with the selected point.

The user interface 110 may distinguish and display a point where a photographed image of the unmanned aerial vehicle 30 is not registered from among a plurality of locations on the cadastral map image im5. The user interface 110 allows the unmanned aerial vehicle 30 to register the captured image in response to selection of a point where the captured image is not registered. In addition, the user interface 110 may provide a search function for images captured by the unmanned aerial vehicle 30 and display at least one captured image im6 of the searched unmanned aerial vehicle 30 .

In some embodiments, the user interface 110 may additionally provide a function of uploading a photographed image of the unmanned aerial vehicle 30 and editing the cadastral map for non-matching parcels between the cadastral map and the current state.

In detail, referring to FIG. 7 , the user interface 110 superimposes the displayed cadastral map image im5 in response to selection of the overlap view affordance so that the photographed image im7 of the unmanned aerial vehicle 30 is overlapped with the cadastral map image im5. Can be displayed overlapping. On the cadastral map image im5, there may be non-overlapping areas where the photographed image of the unmanned aerial vehicle 30 does not exist (regions 277-6 and 712-7 according to the illustrated example). The user interface 110 displays a captured image (im6) in which the selected non-overlapped area is captured in response to selection of one of non-overlapped areas, which is an area where the captured image of the unmanned aerial vehicle 30 does not exist, and selection of an unmanned aerial vehicle image search affordance. ) can be searched and displayed. Referring to FIG. 8 , the user interface 110 may move the captured image im6 according to the selected movement direction in response to selection of the displayed captured image im6 and movement of the selected image. When the captured image im6 moves and is located in the selected non-overlapped area, the user interface 110 registers the captured image im6 and the captured image im7 already overlapped with the cadastral map image im5 to create one image. It can be displayed by overlapping it with the cadastral map image (im5).

Referring to FIG. 9 , the user interface 110 may overlap and display a photographed image im7 of the unmanned aerial vehicle 30 matching a selected point in the displayed cadastral map image im5 with the cadastral map image im5. When the user selects a plurality of points on the cadastral map image (im5), the user interface 110 connects the selected points to each other to set a virtual closed area and converts the closed area into an image (im7) captured by the unmanned aerial vehicle 30. can be filled in. Accordingly, the photographed image im7 and the cadastral map image im5 displayed on the closed area may be displayed overlapping each other.

Referring to FIG. 10 , the computing device 200 may provide a route setting interface 130 capable of setting an autonomous flight route in response to selection of a route setting affordance in the main menu affordance 112 .

The route setting interface 130 may display a map image im8 matched with address information input in the search window 111 . The route setting interface 130 may display a 2D or 3D image of a region matching the searched address in response to selection of the 2D/3D affordance. In addition, the route setting interface 130 may allow the user to change the type of map displayed in response to the selection of the map type affordance.

The route setting interface 130 may display routes connecting the selected plurality of points p1 to p6 on the map image im8. Exemplarily, the user may set a route starting point and a route ending point or passing points through a single selection or a plurality of consecutive selections of the same point on the displayed map image im8. The route setting interface 130 may set the corresponding point as the start point ps of the route when consecutive selections of the same point are received in the displayed map image im8. Further, the route setting interface 130 may set the selected point as a route point in response to selection of a point on the displayed map image im8 (p2 to p5). In addition, when the route setting interface 130 receives continuous selection of the same point in the displayed map image im8, since the starting point ps of the route has already been set, the corresponding point will be set as the ending point pe of the route. can

When the starting point ps and ending point pe of the route are set, the route setting interface 130 may set one flight route pp by connecting all the set points to each other.

The route setting interface 130 may display a video captured while the unmanned aerial vehicle 30 moves along the set flight path pp in response to selection of a photographed image affordance on the route. The route setting interface 130 may display an icon moving along the set flight route pp in synchronization with the reproduction time of the video. The icon here may be an icon resembling the shape of the unmanned aerial vehicle 30, but is not limited thereto.

The user may additionally set the type of work, the altitude, speed, gimbal pitch angle, and moving direction of the unmanned aerial vehicle 30 moving along the set flight path (pp) in the path setting interface 130. let it be In some embodiments, when the speed of the unmanned aerial vehicle 30 is changed, the playback speed of the captured image on the path is also changed accordingly, and the moving speed of the icon moving along the set flight path (pp) may also be changed in synchronization therewith. have.

Referring to FIG. 11 , the path setting interface 130 may provide an environment for setting a work plan within a designated area.

The route setting interface 130 may display a map image im8 matched with address information input in the search window 111 . Further, the route setting interface 130 may automatically set one local area lo by connecting the selected points ap to each other in response to selection of a plurality of points on the displayed map image im8.

The route setting interface 130 may provide area information of the local area lo when the local area lo is set. Also, the route setting interface 130 may provide an interface for selecting the type of work and flight type to be performed by the unmanned aerial vehicle 30 in the local area lo. The user may select one of a plurality of different job types and select one of a plurality of flight types. The route setting interface 130 may automatically set an optimal flight route, flight time, and flight speed within the set regional area lo based on the selected task type and flight type. For example, when the task of spraying the insecticide on the local area lo by the unmanned aerial vehicle 30 is selected, a flight path may be set based on information about the spraying range of the insecticide and the precision of the spraying. In addition, when low-speed flight of the unmanned aerial vehicle 30 is selected as the flight type, the flight speed may be set so that the unmanned aerial vehicle 30 can fly at a preset speed or less.

Referring to FIGS. 12 and 13 , the route setting interface 130 may display a map image im8 matched with address information input in the displayed search window 111 . The path setting interface 130 may display type information im8c of a photographed subject on the displayed map image im8. If rivers, forests, mountains, roads, seas, etc. exist on the map image im8, corresponding information may be displayed.

The route setting interface 130 may receive selection of any one of a plurality of items in the subject type information im8c. For example, the user may select a “river” item. The route setting interface 130 may display the grid map gl in an area matching the selected item on the displayed map image im8 in response to selection of any one of a plurality of items in the subject type information im8c. have. In some embodiments, the grid map gl may be divided into a plurality of partial grids and displayed. A plurality of partial grids may constitute one grid map gl as a whole.

The route setting interface 130 may display total area information of the grid map gl mapped on the map image im8. In addition, the user may select the type of work to be performed by the unmanned aerial vehicle 30 flying on the corresponding grid map gl.

The route setting interface 130 may display the grid map gl in an area that does not overlap with the map image im8 (grid map flight route information). Also, the path setting interface 130 may set a path along which the unmanned aerial vehicle 30 flies along the grid map gl. Also, the route setting interface 130 may display a grid map (egp) for editing, which is a grid map including flight routes.

A default flight path set by default may be displayed on the grid map (egp) for editing. The route setting interface 130 may allow a route to be changed within the range of the grid map egp in response to selection and movement of a point of the flight route displayed on the grid map egp for editing.

In some embodiments, a grid pointer glp may be displayed on each of the plurality of partial grids. The route setting interface 130 displays a partial grid to which the selected grid pointer glp belongs in response to selection of any one of the grid pointers glp displayed on each of the partial grids, and a detailed flight within the displayed partial grid. An interface that enables route setting may be additionally provided.

The route setting interface 130 may display total flight distance and estimated flight time information based on flight route information on the editing grid map (egp).

The computing device 200 may generate autonomous navigation map data based on the flight path set in FIGS. 10 , 11 , and 13 and the map image data on which the flight path is set. The unmanned aerial vehicle 30 may autonomously fly based on autonomous navigation map data and perform autonomous control. In another aspect, the computing device 200 transmits a control command signal of the unmanned aerial vehicle 30 in real time based on the autonomous navigation map data so that the unmanned aerial vehicle 30 can autonomously fly and perform autonomous control according to the control command signal. can make it

According to some embodiments, in FIG. 10 , the route setting interface 130 may automatically set a plurality of layers through which the unmanned aerial vehicle 30 can fly in response to selection of the flight layer setting affordance.

In detail, the computing device 200 may obtain a point cloud of a subject on the ground surface from surface scanning data capturing the ground surface on the currently displayed map image im8 stored in advance. The surface scanning data may be data scanned and collected by a surface scanning device mounted on the unmanned aerial vehicle 30 .

The point cloud that facilitates the autonomous flight of the unmanned aerial vehicle 30 is used as a spatial layer by connecting extracted height values of specific points of an object. Here, the point cloud extracts a height value of a specific point of an object identified using terrain altitude data, and connects the extracted specific height value to determine an area and altitude that facilitates autonomous flight of the unmanned aerial vehicle 30 in space. It is the point of the object to be formed as a layer. In addition, a plurality of layers formed in space are independent of each other, and the layers may be regarded as having a plane horizontal to the ground. Therefore, moving the unmanned aerial vehicle 30 on any one layer means that the altitude of the unmanned aerial vehicle 30 is maintained.

In addition, the embodiment synchronizes the autonomous navigation map for the flight of the unmanned aerial vehicle 30 built in the layer with the unmanned aerial vehicle 30 within a safety standard through GPS or information from a position coordinate correction device, so that the unmanned aerial vehicle 30 It can be formed as an applicable spatial map.

In some embodiments, a layer having a certain altitude value from the ground surface on which the unmanned aerial vehicle 30 can fly is set according to a mission for the unmanned aerial vehicle 30, and a flight path of the unmanned aerial vehicle 30 on the set layer. , and an autonomous navigation map including set layers and flight path information on the layers can be built. Here, the route may be composed of at least two or more waypoints including the location of ground objects existing on the surface of the route. In addition, an autonomous navigation map for each mission may be constructed according to the identification information of the unmanned aerial vehicle 30 . The waypoint is a point at which the mission assigned to the unmanned aerial vehicle 30 can be performed. Accordingly, by providing an autonomous flight map in an invisible area, it is possible to overcome the limitations of operation within the pilot's visual range for areas where it is difficult to maintain a constant altitude value with ground objects. In addition, a map for the flight of the unmanned aerial vehicle 30 in which altitude values are reflected may be produced by shaping the autonomous flight space into layers from ground scanning and image capture data and matching the data to the shaped layers.

In some embodiments, specific path information of a flight path, which is an airway on a layer, may be determined as one of various flight paths predetermined according to tasks assigned to the unmanned aerial vehicle 30 . In addition, even if it is determined as one of various flight routes, it may be allowed for the user to directly set details of the route through the route setting interface 130 .

Meanwhile, the computing device 200 uses a Digital Surface Model (DSM) or a Digital Terrain Model (DTM) among terrain elevation data to obtain a height value of a specific point of the subject identified by the identification unit. can be extracted. DSM data and DTM data are data that can be acquired from a government agency (for example, the National Geographic Information Institute in Korea) or an aerial surveying company that is constructing and constructing a database of geographic information of each country. The computing device 200 connects the height value of a specific point of the extracted subject to shape the area and altitude at which the unmanned aerial vehicle 30 can autonomously fly in space as a layer, and the unmanned aerial vehicle 30 in the layer shaped in the space ( 30) Based on at least one or more of information on the type of task assigned to, flight altitude limit data, precise numerical map, and route information to avoid the military security area or no-fly zone, or by matching them, the flight of the unmanned aerial vehicle in space By setting a specific flight path for each flight, it is possible to build a layer that composes an autonomous navigation map.

The data of the autonomous navigation map includes information on a plurality of layers in an area where the unmanned aerial vehicle 30 will fly autonomously. For example, information on the first and second layers at different altitudes and flight path information of the unmanned aerial vehicle 30 on each layer may be mapped. Illustratively, along the flight path of the first layer, the first unmanned aerial vehicle may autonomously fly while maintaining the altitude according to the first layer, and along the flight path of the second layer, the second unmanned aerial vehicle may fly along the flight path of the second layer. It can fly autonomously while maintaining altitude. The flight path information herein may be optimal path information for performing tasks assigned to the unmanned aerial vehicle 30 .

According to some embodiments, the path setting interface 130 in FIG. 10 may display the obstacle avoidance simulation interface 140 as shown in FIG. 14 in response to selection of the obstacle avoidance simulation affordance.

FIG. 14 is for explaining simulation of avoiding obstacles by an unmanned aerial vehicle in a state in which the setting of the straight-line distance between route points to be described in FIG. 15 and the setting of the flight route in the above-described route setting interface are completed.

Referring to FIG. 14 , the first obstacle avoidance simulation interface 140a may display various obstacle information in response to selection of an obstacle type selection affordance, and may display a selected obstacle in response to selection of one of the displayed obstacle items. can Also, the first obstacle avoidance simulation interface 140a may change the position of the obstacle in response to the displayed obstacle selection and movement of the selection. The first obstacle avoidance simulation interface 140a maintains autonomous flight while avoiding moving to another layer when an unidentified obstacle exists on the autonomous navigation map on the flight path of the unmanned aerial vehicle 30 in response to the selection of the simulation execution affordance. You can run a simulation to show that

As shown in the first obstacle avoidance simulation interface 140a, a plurality of first path points LP1 forming a flight path on the first layer L1 are shown. When the first path points LP1 are connected to each other, a first flight path on the first layer L1 is formed. In addition, a plurality of second path points LP2 forming a flight path on the second layer L2 are shown, and when the second path points LP2 are connected to each other, the second flight on the second layer L2 becomes a path When the unmanned air vehicle 30 autonomously flies along the second flight path on the second layer L2 where its flight path is set, and detects an obstacle (a tree in the illustrated example) in front, the second layer L2 After ascending to the first layer (L1), which is a higher layer at a higher altitude, flying along the route point in the upper layer, descending to the first layer (L1) again, and then reaching the route point on the first layer (L1) It can fly autonomously along the flight path of When the unmanned aerial vehicle 30 autonomously flies on the first layer L1, which is a layer not assigned to itself, the second path point LP2 on the second layer L2 is located on the first layer L1 in the vertical direction. You can fly along the replacement points (RPs) mapped to. That is, when the unmanned aerial vehicle 30 reaches the N-th second path point of the second layer (L2) and then ascends to the first layer (L1), the N+1-th second path point is mapped in the vertical direction. It may move to the replacement point RP on the first layer L1. Here, the N-th second path point may be the first second path point that the unmanned aerial vehicle 30 will pass through at the time of recognizing the obstacle. Also, the unmanned aerial vehicle 30 may move on the first layer L1 along the remaining replacement points RP. And, when the unmanned aerial vehicle 30 descends to the second layer L2 after reaching the last replacement point of the first layer L1, the second path on the second layer L2 to which the last replacement point is mapped. It is possible to move to a second path point next to the point. In addition, the unmanned aerial vehicle 30 may autonomously fly along the remaining second path points on the second layer L2. In some embodiments, when the drone 30 moves along the replacement point RP on the first layer L1, the obstacle detected when the drone 30 moves on the second layer L2 is no longer on its lower side. It can be simulated to move along the replacement point RP until it is determined that it is not detected in the direction. In the embodiment, when recognizing an obstacle that can be treated as an obstacle on the flight path due to a sudden obstacle, an obstacle that is not reflected on the autonomous navigation map, or a subject whose shape or size is variable although it was reflected on the autonomous navigation map . On the other hand, when the unmanned aerial vehicle 30 recognizes an obstacle, it has been described as moving to an upper layer, but is not limited thereto and may move to a lower layer. In this case, the path point on the original moving layer is located in the lower direction. A replacement point may be set on the lower layer by being mapped on the lower layer.

When a plurality of unmanned aerial vehicles autonomously fly in each of a plurality of layers, the obstacle avoidance simulation interface 140 may provide a simulation of correction of a flight path of another unmanned aerial vehicle during an obstacle avoidance flight.

As shown in the second obstacle avoidance simulation interface 140b, the first unmanned aerial vehicle 30a autonomously flies along the first path point LP1 on the first layer L1, and the second layer L2 Assuming that the second unmanned aerial vehicle 30b autonomously flies along the second path point LP2, as described in the first obstacle avoidance simulation interface 140a, when the second unmanned aerial vehicle 30b recognizes an obstacle, it Obstacles may be avoided while flying along the replacement point RP on the unallocated first layer L1. In this case, the flight path of the first unmanned aerial vehicle 30a autonomously flying along the first path point LP1 on the first layer L1 may be partially changed. A risk area layer DA may be set on the first layer L1. The risk area layer DA may overlap the first layer L1 and may be a virtual plane passing through the replacement point RP. Also, the danger area layer DA may be set such that all boundary points of the danger area layer DA are spaced apart from all replacement points RP by a predetermined horizontal distance. Among all the first path points LP1 on the first layer L1, the first path point overlapping the risk area layer DA is mapped to the upper layer L0 of the first layer L1 and is mapped to the upper layer L0. ), another replacement point that replaces the first path point may be set. Also, the first unmanned aerial vehicle 30a may move to the upper layer L0 and fly along the replacement point. The method of setting the alternative point on the upper layer (L0) is the same as described in the first obstacle avoidance simulation interface 140a, and the way the first unmanned aerial vehicle 30a flies is also described in the first obstacle avoidance simulation interface 140a. (30) can be applied in the same way as flying. In the embodiment, even if an obstacle is not recognized on the flight path on the first layer L1 of the first unmanned aerial vehicle 30a, another unmanned aerial vehicle in its own layer L1 according to the avoidance function of the unmanned aerial vehicle on another layer. When an air vehicle enters and flies, a mutual collision problem may occur or another unmanned air vehicle may be recognized as an obstacle. Therefore, while allowing the first unmanned aerial vehicle 30a to also perform an avoidance function, the range of path points of the first unmanned aerial vehicle 30a to be mapped is determined in consideration of the range recognized as a danger area, thereby solving the problem of collision with other unmanned aerial vehicles. etc. can be prevented.

The user can test the stability of the set flight path through simulation, and can modify the flight path or the spacing between path points as needed.

Each of the first and second obstacle avoidance simulation interfaces 140a and 140b responds to the selection of the flight path determination affordance, and provides data on how to set alternative points when recognizing obstacles, and settings of a danger area layer DA when a plurality of unmanned aerial vehicles fly. Automatic navigation map data including method data is generated, and the unmanned aerial vehicle 30 is allowed to autonomously fly based on the automatic navigation map data.

15 illustrates an interface for a method of recognizing an obstacle, avoiding flight to another layer, and setting distance information between route points.

The flight path on the layer shown in FIG. 15 is the flight path on the first layer set without recognizing the obstacle.

In FIG. 10, the route setting interface 130 automatically forms a plurality of layers through which the unmanned aerial vehicle 30 can fly in response to the selection of the flight layer setting affordance, and then obtains distance information between route points as shown in FIG. 15. An interface 150 for setting may be displayed.

Referring to FIG. 15, a flight path on a layer (L) may be composed of a plurality of path points. The unmanned aerial vehicle 30 flies along the flight path while passing through the path point LP on the layer L assigned thereto. In some embodiments, the maximum separation distance between the closest path points is the preset maximum speed of the unmanned aerial vehicle 30, the preset unmanned aerial vehicle 30 at the time when the preset unmanned aerial vehicle 30 recognizes the presence of an obstacle in front, and It may be determined based on the minimum separation distance value rl between obstacles. Here, the minimum separation distance value rl is a value that varies depending on the performance of the unmanned aerial vehicle 30, and may be determined in advance by considering the performance of the unmanned aerial vehicle 30 itself when setting the autonomous navigation map.

Referring to (a) of FIG. 15 , flight distances (l1, l2, l3, l4, l5, l6) from an arbitrary route point to the next route point may continuously vary. However, a right triangle having the hypotenuse of the flight distance (l1, l2, l3, l4, l5, l6) between the nearest path points may be considered. At this time, the bases of all right triangles can be x1. In some embodiments, the path points may be set to have the same base value of all right triangles. Then, among the two closest path points in the right triangle, a path point corresponding to the starting point is defined as a vertex, and an angle of the vertex is defined as tehta. In addition, x1 here may be a value of “flying distance of the UAV between points of the shortest combined path” * “cos(theta)”. In other words, it can be defined as the product of “the flight distance of the UAV between the most inclusive path points” and cos(theta).

The distance between the unmanned aerial vehicle 30 and the obstacle when the unmanned aerial vehicle 30 recognizes the obstacle is defined as rl. rl may be defined as a distance between the unmanned aerial vehicle 30 and an obstacle in the longitudinal direction and the horizontal direction of the layer (L).

It is assumed that the unmanned aerial vehicle 30 must fly avoidance before reaching the minimum collision avoidance distance ep between the unmanned aerial vehicle 30 and the obstacle for efficient avoidance of the obstacle. The minimum anti-collision distance ep may be defined as a distance between the unmanned aerial vehicle 30 and an obstacle in the longitudinal direction and the horizontal direction of the layer L.

If an obstacle is recognized while the unmanned aerial vehicle 30 is moving along a path corresponding to the 11 flight distance, avoidance flight may be performed at a path point to be passed through in the future. Since the straight-line distance between the path point at which the avoidance flight is initiated and the obstacle is larger than the minimum collision avoidance distance (ep), the avoidance flight can be safely performed without a collision problem between the unmanned aerial vehicle 30 and the obstacle.

Referring to (b) of FIG. 15 , it can be seen that x2, which is the value of “flight distance of the UAV between the closest merge path points” * “cos(theta)”, has a larger value than x1 in (a). In this case, if the unmanned aerial vehicle 30 recognizes an obstacle while moving on a path corresponding to the 11 flight distance, avoidance flight may be performed at a path point to be passed through in the future. Since the straight-line distance between the path point at which the avoidance flight is initiated and the obstacle is smaller than the minimum collision avoidance distance (ep), the obstacle collision problem occurs when the unmanned aerial vehicle 30 makes an avoidance flight from the path point at which the avoidance flight is initiated to an upper or lower layer. can happen Therefore, by limiting the maximum value of the "flying distance of the UAV between the closest matching path points" * "cos(theta)", safe avoidance flight against unexpected obstacles is possible. The interface 150 for setting distance information between route points allows the user to adjust the minimum distance (x1) and maximum distance (x2) between the nearest route points. The interface 150 for setting the distance information between route points configures a preset flight route based on the minimum distance (x1) and maximum distance (x2) between the nearest route points set in response to the selection of the simulation affordance. The distance between them is adjusted, and the minimum separation distance value (rl) between the unmanned aerial vehicle 30 and the obstacle at the time when the unmanned aerial vehicle 30 recognizes the presence of an obstacle in front, the minimum anti-collision distance (ep), and A flight route, etc. is displayed so that the user can check the maximum and minimum separation distance between route points. Users can set detailed flight routes based on maximum and minimum separation distances. Therefore, it is possible to increase the degree of freedom in setting a flight path while considering sudden obstacles.

Embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. medium), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes generated by a compiler. A hardware device may be modified with one or more software modules to perform processing according to the present invention and vice versa.

Specific implementations described in the present invention are examples and do not limit the scope of the present invention in any way. For brevity of the specification, description of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection of lines or connecting members between the components shown in the drawings are examples of functional connections and / or physical or circuit connections, which can be replaced in actual devices or additional various functional connections, physical connection, or circuit connections. In addition, if there is no specific reference such as “essential” or “important”, it may not be a component necessarily required for the application of the present invention.

In addition, the detailed description of the present invention described has been described with reference to preferred embodiments of the present invention, but those skilled in the art or those having ordinary knowledge in the art will find the spirit of the present invention described in the claims to be described later. And it will be understood that the present invention can be variously modified and changed without departing from the technical scope. Therefore, the technical scope of the present invention is not limited to the contents described in the detailed description of the specification, but should be defined by the claims.

Claims (6)

Displaying a user interface by executing a ground control program for autonomous flight and autonomous control of an unmanned aerial vehicle executed on a computing device;
displaying a map image matching the received address information and a plurality of work areas and work information overlapping the map image; and
Including; displaying control status, flight path, and state information of the unmanned aerial vehicle of the selected work area among the plurality of work areas,
displaying at least one of image information obtained by capturing the selected work area while the UAV is flying, route information on a path the UAV has already flown along a flight path, and information on a current location of the UAV within the selected work area;
displaying a route setting interface displaying a map image in response to selection of a route setting affordance on the user interface; and
In response to selection of a plurality of points on the map image displayed on the route setting interface, setting a flight path connecting the selected points, and reproducing the captured image while the UAV moves along the set flight path; do,
The reproduction speed of the image is changed based on the flight speed of the unmanned aerial vehicle set on the route setting interface;
displaying type information of a photographed subject on a map image displayed on the route setting interface;
displaying a grid map in an area matching the selected item in the map image displayed on the route setting interface in response to selection of any one of a plurality of items in the type information;
displaying a grid map for editing allowing the flight path of the unmanned aerial vehicle to be set and modified within the grid map; and
The grid map is composed of a plurality of partial grids, and providing an interface allowing setting and modification of detailed flight paths in the selected partial grid in response to selection of any one of the plurality of partial grids; further comprising
A method of performing autonomous control using an unmanned aerial vehicle capable of autonomous flight. According to claim 1,
displaying, in the user interface, a cadastral map image matching the searched address in response to selection of the cadastral map affordance; and
Responding to the selection of a point on the displayed cadastral map image, displaying an image taken by an unmanned aerial vehicle of the selected point superimposed on the selected point;
A method of performing autonomous control using an unmanned aerial vehicle capable of autonomous flight. According to claim 1,
displaying, by the user interface, a cadastral map image matching the searched address in response to selection of the cadastral map affordance, and overlapping and displaying an image of a region on the cadastral map image on which the unmanned aerial vehicle is displayed; and
Responding to selection of an area where the captured image and the cadastral map image do not overlap, loading an image of a selected point and matching the selected area to the non-overlapping area
A method of performing autonomous control using an unmanned aerial vehicle capable of autonomous flight. delete delete Displaying a user interface by executing a ground control program for autonomous flight and autonomous control of an unmanned aerial vehicle executed on a computing device;
displaying a map image matching the received address information and a plurality of work areas and work information overlapping the map image; and
Including; displaying control status, flight path, and state information of the unmanned aerial vehicle of the selected work area among the plurality of work areas,
displaying at least one of image information obtained by capturing the selected work area while the UAV is flying, route information on a path the UAV has already flown along a flight path, and information on a current location of the UAV within the selected work area;
displaying a route setting interface displaying a map image in response to selection of a route setting affordance on the user interface; and
The route setting interface further comprises setting a plurality of layers through which the unmanned aerial vehicle can fly in response to selection of a flight layer setting affordance;
The setting of the plurality of layers may include obtaining a point cloud of objects from surface scanning data, identifying the objects based on the analysis of the point cloud, and connecting specific points of the objects based on the analysis of the point cloud. By setting a plurality of layers to set a flight path that allows the unmanned aerial vehicle to autonomously travel on any one of the plurality of layers,
When the UAV recognizes an obstacle on the flight path, avoid moving to a layer other than the layer assigned to the UAV, but maintain the shape of the flight path on the layer assigned to the UAV and simulate flying on the other layer providing an obstacle avoidance simulation interface; further comprising
A method of performing autonomous control using an unmanned aerial vehicle capable of autonomous flight.

Images