KR20230030074A - Method and apparatus for controlling drone landing - Google Patents

Abstract

Provided is a method for controlling the landing of a drone (201), comprising: a step of recognizing a target (203) by using an ultrasonic sensor and a GPS sensor of the drone (201); a step of detecting a first position of the target (203); a step of acquiring an RGB image and an IR image of the target (203) by using an optical sensor and an IR sensor of the drone (201); a step of recognizing the target (203) as a landing target (203) for the drone (201) to land on from the RGB image and the IR image by using a recognizer trained to recognize targets (203) for the drone (201) to land on from the RGB images and the IR images; a step of detecting a second position of the target (203) which is the landing target (203); a step of generating a precise position of the target (203) by using the first position and the second position; and a step of controlling the landing of the drone (201) based on the precise position of the target (203) which is the landing target (203) and the position of the drone (201). Therefore, landing safety can be increased.

Description

Method and device for controlling drone landing {METHOD AND APPARATUS FOR CONTROLLING DRONE LANDING}

The present invention relates to a method, apparatus and system for controlling the landing of a drone (201).

A drone is an air vehicle that flies by remote control or autonomously along a designated path, and is used in various fields such as camera shooting, military use, transportation, security, and personal use. Drones are generally used as small unmanned aerial vehicles that are manually operated by an operator's radio operation signal, and manned drones are also commercialized.

Although the range of use of drones is gradually expanding, landing errors of drones can lead to fatal accidents and can cause fatal damage in drone operation. Therefore, research on technology for controlling the landing of drones more accurately and safely is required.

Embodiments seek to save the time it takes for a drone to recognize a target to land on and the load required for target recognition.

Embodiments are intended to increase the safety of landing by increasing the recognition accuracy of a target on which a drone will land.

A landing control method for a drone 201 according to an embodiment includes the steps of recognizing a target 203 and detecting a first position of the target 203 using an ultrasonic sensor and a GPS sensor of the drone 201; Acquiring an RGB image and an IR image of the target 203 using an optical sensor and an IR sensor of the drone 201; The drone 201 identifies the targets 203 from the RGB images and the IR images by using a recognizer learned to recognize the targets 203 on which the drone 201 will land from the RGB images and the IR images. Recognizing a landing target 203 to land on and detecting a second position of the target 203, which is the landing target 203; generating a precise position of the target 203, which is a landing target 203, using the first position and the second position; Recognizing a direction using a geomagnetic sensor and setting a landing direction; and controlling the landing of the drone 201 based on the precise position of the target 203 as the landing target 203, the landing direction, and the position of the drone 201.

According to an embodiment, the detecting of the second position of the target 203 may include generating at least one feature corresponding to at least one pre-learned recognizer based on the RGB image and the IR image. ; determining whether the target (203) is a landing target (203) based on at least one output produced by the at least one recognizer from the at least one feature; and detecting a second position corresponding to at least one of the RGB image and the IR image when the target 203 is the landing target 203 .

According to an embodiment, generating the feature may further include generating at least one feature corresponding to the recognizer based on information obtained from the ultrasonic sensor and the GPS sensor.

According to an embodiment, the detecting of the second position of the target 203 may include at least one image corresponding to at least one pre-learned first recognizer 301 based on the RGB image and the IR image. generating a first feature (321); based on at least one first output (322) produced by the first recognizer (301) from the first feature (321), determining whether the target (203) is a landing target (203); When the target 203 is the landing target 203, at least one second feature 323 corresponding to the pre-learned at least one second recognizer 302 is generated based on the first output 322 doing; and based on at least one second output (324) produced by the second recognizer (302) from the second feature (323), generating a second location of the target (203), which is a landing target (203). and, for learning by the first recognizer 301, first training features are generated from training RGB images and training IR images, and the first recognizer 301 generates the training RGB images and the It is learned based on first labels corresponding to training IR images and first training outputs generated by the first recognizer 301 from the first training features, and learned by the second recognizer 302. , second training features are generated from the first training outputs, and the second recognizer 302 uses second labels corresponding to the second training outputs and the second training features from the second training features. It can be learned based on the second training outputs generated by recognizer 302 .

According to an embodiment, the step of controlling the landing of the drone 201 includes at least one corresponding to at least one pre-learned third recognizer 303 based on the precise location and the location of the drone 201. generating a third feature (325) of; and generating control information for landing of the drone 201 based on at least one third output 326 generated by the third recognizer 303 from the third feature 325. , third training features are generated from the training precise positions and positions of the drone 201, for learning by the third recognizer 303, the third recognizer 303 combines the training precise positions and the It can be learned based on third labels corresponding to locations and third training outputs generated by the third recognizer 303 from the third training features.

According to one embodiment, the generating of the precise position comprises at least one identifier corresponding to at least one recognizer learned to recognize the precise positions of the landing targets 203, based on the first position and the second position. creating features; and generating a precise location of the target 203 based on at least one output the recognizer produces from the feature.

A landing control method of a drone 201 according to an embodiment includes transmitting the generated precise location to the GCS 204 of the target 203; receiving an offset generated by the GCS (204) based on the reference position and the precise position of the target (203); receiving a landing authorization signal generated by the GCS (204) in response when the offset meets a predefined range; and correcting the precise position based on the offset, wherein the step of controlling the landing of the drone 201 is based on the corrected precise position and the position of the drone 201, (201) may include generating control information for landing.

A landing control method of a drone 201 according to an embodiment includes receiving a reference position of the target 203 from a GCS; based on the reference position and the precise position, generating an offset; transmitting the offset to the GCS; and receiving a landing permission signal generated by the GCS in response to a case where the offset satisfies a predefined range.

A landing control method of a drone 201 according to an embodiment includes transmitting a reference position of the target 203 to a GCS; receiving a result of determining whether the reference position has changed from the GCS; generating an offset based on the reference position and the precise position when there is no change in the reference position; and if there is a change in the reference position, generating an offset based on the reference position received from the GCS and the precise position.

A landing control method for a drone 201 according to an embodiment includes receiving a precise position of a target 203 generated from a drone 201; based on the reference position and the precise position of the target (203), generating an offset; determining whether the offset satisfies a predefined range; and transmitting a landing permission signal to the landing control device of the drone 201 based on the determination result.

A landing control method of a drone 201 according to an embodiment includes information obtained from an ultrasonic sensor and a GPS sensor of the drone 201, an RGB image of a target 203 obtained from an optical sensor and an IR sensor of the drone 201, and generating at least one feature based on at least one of the IR images; and recognizing the target 203 as a landing target 203 on which the drone 201 will land, based on the characteristics, using a recognizer learned to recognize the targets 203 on which the drone 201 will land, and landing and generating at least one of a precise location of the target 203 and control information for landing of the drone 201 .

An apparatus according to an embodiment may be combined with hardware and controlled by a computer program stored in a medium to execute any one of the methods described above.

Embodiments can save the time it takes for a drone to recognize a target to land on and the load required for target recognition.

Embodiments may increase the safety of landing by increasing the recognition accuracy of the target on which the drone lands.

1 is a flowchart illustrating a drone landing control method according to an embodiment.
2 is a conceptual diagram for explaining a drone landing control method according to an embodiment.
3 is a diagram for explaining a drone landing control method according to an embodiment.
4 is a diagram for explaining a drone landing control method according to an embodiment.
5 is a diagram for explaining a learning method according to an exemplary embodiment.
6 is a flowchart illustrating a drone landing control method according to an embodiment.
7 is a flowchart illustrating a drone landing control method according to an embodiment.
8 is an exemplary diagram of a configuration of a device according to an embodiment.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only, and may be modified and implemented in various forms. Therefore, the embodiments are not limited to the specific disclosed form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Although terms such as first or second may be used to describe various components, such terms should only be construed for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element.

It should be understood that when an element is referred to as being “connected” to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, but one or more other features or numbers, It should be understood that the presence or addition of steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this specification, it should not be interpreted in an ideal or excessively formal meaning. don't

The embodiments may be implemented in various types of products such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent vehicles, kiosks, and wearable devices. For example, embodiments may be applied to recognizing a user in a smart phone, mobile device, smart home system, and the like. The embodiments may be applied to a payment service through user recognition. In addition, the embodiments may be applied to an intelligent vehicle system that recognizes a user and automatically starts the engine. Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Like reference numerals in each figure indicate like elements.

1 is a flowchart illustrating a drone landing control method according to an embodiment.

Referring to FIG. 1 , the drone landing control apparatus according to an embodiment may recognize a target and detect a first position of the target using an ultrasonic sensor and a Global Positioning System (GPS) sensor of the drone (101). . The drone landing control device is a device for controlling the landing of a drone, and may be implemented as a server or a terminal, and may be implemented as, for example, a software module, a hardware module, or a combination thereof. The drone landing control device may be mounted on a drone and process a signal, command, or information for controlling the landing of the drone. The drone landing control device may process a signal, command, or information for controlling the landing of a drone outside of a drone to be controlled, and control the drone while communicating with the drone remotely. The drone may automatically perform flight operations, take-off and landing operations, information collection operations, drone status information acquisition operations, and charging operations according to input or received commands or according to manual adjustment. Not limited.

The drone landing control device may recognize a target based on information obtained from an ultrasonic sensor and a GPS sensor of the drone. Here, the target may be a landing target where the drone wants to land. The target may be a ground control station (GCS) landing target, and may be displayed as a marker having a predefined shape, for example. The drone landing control device may recognize a target based on a predefined marker.

According to an embodiment, the GCS may transmit or receive commands, plans, and information for drone operation through communication with external GCSs or drones. A network of a system for operating drones enables wired and wireless communication between various entities in the system. GCSs and drones may communicate with each other over a network, and the network may use standard communication technology and/or protocols. The drone and the GCS can guide the landing of the drone accurately, safely and quickly by exchanging information to control the landing of the drone.

The drone landing control device may recognize a positional relationship between a target and a drone based on information acquired from an ultrasonic sensor and a GPS sensor of the drone. The drone landing control device may recognize the location of the drone based on information acquired from the GPS sensor. The drone landing control device may measure the distance between the target and the drone based on information acquired from the ultrasonic sensor and the GPS sensor of the drone. The drone landing control device may recognize the first location of the target based on the location of the drone. Here, the first position is the position of the target detected from information obtained from the ultrasonic sensor and the GPS sensor.

Referring to FIG. 1 , the drone landing control device may obtain an RGB image and an IR image of a target using an optical sensor and an infrared ray (IR) sensor of a drone (102). The drone landing control device may obtain an RGB image and an IR image of a target recognized from an ultrasonic sensor and a GPS sensor. The drone landing control device quickly recognizes a target from an ultrasonic sensor and a GPS sensor, takes an image of the recognized target, increases the recognition speed of the target, saves the load required for target recognition, and operates necessary for target recognition. can increase their processing efficiency. The drone landing control device supplements the position detection of the target based on the ultrasonic sensor with the position detection based on the optical sensor and the IR sensor, thereby increasing target recognition accuracy and detecting the target position more precisely.

Referring to FIG. 1, the drone landing control device uses a recognizer learned to recognize targets on which the drone will land from RGB images and IR images, and selects a target from the acquired RGB images and IR images as a landing target on which the drone will land. , and the second position of the target, which is the landing target, can be detected (103). Here, the second position is the position of the target detected based on the RGB image and the IR image. The drone landing control apparatus may identify a target from the acquired RGB image and IR image, and detect a second position corresponding to the identified target's RGB image and IR image.

The drone landing control device may determine whether a target is a landing target using a pre-learned recognizer and detect a second position of the target. An embodiment using the recognizer will be described later with reference to FIGS. 3 and 4 . The recognizer can be learned by a deep learning technique, and an embodiment related to learning will be described later with reference to FIG. 5 .

Referring to FIG. 1, the drone landing control device uses a first position detected based on an ultrasonic sensor and a GPS sensor and a second position detected based on an optical sensor and an IR sensor to determine the precise position of a target as a landing target. can be created (104). According to an embodiment, the drone landing control device may correct the first position using the second position and generate the precise position of the target based on the corrected result. The accuracy of the second location may be higher than that of the first location, and in this case, the drone landing control device may increase the accuracy of detecting the location of the target by using the second location.

According to an embodiment, the drone landing control device may generate the precise location of the target from the first location and the second location using the pre-learned recognizer. The recognizer can be taught to recognize precise positions of landing targets from positions detected by ultrasonic and GPS sensors and positions detected by optical and IR sensors. The drone landing control device may generate at least one feature corresponding to at least one pre-learned recognizer based on the first location and the second location. The drone landing control device may generate a precise position of the target based on at least one output generated by at least one pre-learned recognizer from at least one feature. A method of generating a precise position from a first position detected based on an ultrasonic sensor and a GPS sensor and a second position detected based on an optical sensor and an IR sensor using a machine learning technique includes aspects of a neural network, dimensions of input and output , feature definition method, learning technique, etc. can be applied in various ways.

According to one embodiment, the drone landing control device may determine whether a difference between the first position and the second position exceeds a predefined threshold range. The drone landing control apparatus may repeatedly detect at least one of the first location and the second location when the second location is farther away from the first location by exceeding a threshold range. The drone landing control device may correct the first position by using the second position when a difference between the detected first position and the second position is within a predefined threshold range. The drone landing control device may generate the precise position of the target from the calibration result.

According to an embodiment, the drone landing control device may apply predefined weights to the first position and the second position, respectively, and generate a precise position of the target based on a weight application result. Here, weights may be set based on a difference between the first position and the second position. The drone landing control apparatus may use a table recording weights corresponding to differences between the first position and the second position. The drone landing control apparatus may obtain weights to be applied to the first position and the second position, respectively, from a table databased in advance.

Referring to FIG. 1 , the drone landing control device may further include a geomagnetic sensor. The geomagnetic sensor can measure the direction of the drone, determine whether it is the intended landing direction of the drone, and if there is a difference between the direction of the drone and the intended landing direction of the drone, by controlling the rotation of the drone, You can land the drone in the set direction.

Referring to FIG. 1 , the drone landing control device may control the landing of the drone based on the precise location of the landing target and the location of the drone (105). The drone landing control device may detect the location of the drone using a GPS sensor. The drone landing control device may determine a positional relationship between the drone and the target based on the position of the drone and the precise position of the target. The drone landing control device may generate control information for landing of the drone based on the positional relationship between the drone and the target and the landing direction. The drone landing control device may control actuators of the drone using control information.

According to an embodiment, the drone landing control device may generate control information for landing of the drone from the precise location of the target and the location of the drone using a pre-learned recognizer. An operation of generating control information using the recognizer will be described later with reference to FIGS. 3 and 4 .

The drone landing control device may utilize information obtained through communication with the GCS in generating control information. Details related to communication between the drone landing control device and the GCS will be described later with reference to FIG. 2 .

The drone landing control device may obtain environmental information related to flight or landing of the drone. Environmental information is information related to the environment that may affect the flight or landing of the drone, and includes, for example, parameters such as weather information (wind speed, rainfall, temperature and humidity, etc.) and information on drone flight availability. The drone landing control device may obtain environmental information from AWS (Automatic Weather System). AWS can be installed on drones or GCS. The drone landing control device may obtain environmental information from an external server. The drone landing control device may obtain environmental information through communication with the GCS or an external server. The drone landing control apparatus may generate control information for landing of the drone based on the precise location of the target, the location of the drone, and environment information. Alternatively, the drone landing control device may correct pre-generated control information based on environmental information. The drone landing control device may generate at least one feature corresponding to at least one pre-learned recognizer based on the precise location of the target, the location of the drone, and environment information. The drone landing control device may obtain control information generated by at least one pre-learned recognizer from at least one feature.

2 is a conceptual diagram for explaining a drone landing control method according to an embodiment.

The drone landing control device (not shown) may recognize the location of the target 203 based on information obtained from the sensors 202 of the drone 201 . Sensors 202 may include ultrasonic sensors, GPS sensors, optical sensors, and IR sensors. The drone landing control device may control the landing of the drone 201 through communication with the GCS 204 .

Referring to FIG. 2(a), the drone landing control device may recognize a target 203 using an ultrasonic sensor and a GPS sensor and detect a first position of the target 203.

Referring to FIG. 2(b), the drone landing control device may generate features from an RGB image and an IR image acquired using an optical sensor and an IR sensor, and apply the generated features to a pre-learned recognizer. The drone landing control device may determine whether the target 203 is a landing target for the drone 201 to land on, based on the output of the recognizer. When the target 203 is a landing target, the drone landing control device may detect a second position corresponding to the RGB image and the IR image.

Referring to FIG. 2(c) , the drone landing control device may control the landing of the drone 201 using the precise position of the target generated using the first position and the second position. The drone landing control device may transmit the generated precise location to the GCS 204 of the target 203 . The GCS 204 may generate an offset based on the pre-stored reference position of the target 203 and the received precise position.

The precise position may be expressed as (x1, y1, z1), and the pre-stored reference position may be expressed as (x2, y2, z2). According to one embodiment, the GCS 204 may generate an offset using a difference between the precise position (x1, y1, z1) and the reference position (x2, y2, z2) or a value based on the difference. According to an embodiment, the GCS 204 generates feature values by pre-processing the precise position (x1, y1, z1) and the reference position (x2, y2, z2), respectively, and Lp-norm based on the generated feature values. Calculate , and create an offset based on the calculated value. Various schemes may be employed for generating the offset.

The GCS 204 may transmit a landing permission signal to the drone landing control device in response to a case where the generated offset satisfies a predefined range. The drone landing control device may continue controlling the landing of the drone 201 in response to the landing permission signal. The drone landing control device may correct the precise position based on the offset received from the GCS 204. The drone landing control device may generate control information for landing of the drone 201 based on the corrected precise position and the position of the drone 201 . The control information may be transmitted to the actuator of the drone 201, and may include, for example, values of (Pitch, Roll, Yaw), landing speed, and the like.

According to one embodiment, the drone landing control device may receive the reference position of the target 203 from the GCS 204 . The drone landing control device may generate an offset based on the received reference position and precise position. The drone landing control device may transmit the offset to the GCS 204. As described above, the GCS 204 may transmit a landing permission signal to the drone landing controller in response to the case where the offset satisfies a predefined range. The drone landing control device may perform correction of an offset-based precision position in response to a landing hunger signal, and may transmit control information for landing.

According to one embodiment, the drone landing control device may transmit the previously stored reference position of the target 203 to the GCS 204 . The GCS 204 may determine whether the received reference position is changed. The GCS 204 may transmit a determination result on whether the reference position is changed to the drone landing control device. When the reference position is changed, the GCS 204 may transmit the changed reference position to the drone landing control device. When there is no change in the reference position, the drone landing control device may generate an offset based on the reference position and the precision position. When there is a change in the reference position, the drone landing control device may generate an offset based on the reference position and the precise position received from the GCS 204 .

The drone landing control device may generate control information for landing of the drone from the precise position and offset using the pre-learned recognizer. Here, the recognizer can be taught to generate control information from precise positions and offsets. The drone landing control device may generate at least one feature from the precise position and offset. The drone landing control device may apply at least one generated feature to a pre-learned recognizer. The drone landing control device may generate control information for landing of the drone based on at least one output generated by the pre-learned recognizer.

The above-described operations, such as generating an offset for the precise position of the drone, generating control information for landing of the drone, and generating a landing permission signal, are performed by the drone landing control device or by the GCS 204. Alternatively, the drone landing control device and the GCS 204 may be interlocked. Subjects of the above-described operations may be variously modified according to system efficiency, design intention, or load burden policy.

Referring to FIG. 2(d) , the drone landing control device may complete the landing of the drone 201 by controlling the landing of the drone 201 based on the generated control information.

3 is a diagram for explaining a drone landing control method according to an embodiment.

Referring to FIG. 3, the drone landing control device uses a first recognizer 301, a second recognizer 302, and a third recognizer 303 to obtain a landing target from an RGB image 311 and an IR image 312. It is possible to determine whether or not the target is detected, recognize the location (x, y, z) of the target, and generate control information (pitch, roll, yaw) for landing.

The drone landing control apparatus may generate at least one first feature 321 corresponding to at least one first recognizer 301 previously learned from the RGB image 311 and the IR image 312 . The drone landing control device may additionally utilize the sensor information 313 in generating the first feature 321 . The sensor information 313 includes information obtained from an ultrasonic sensor and a GPS sensor.

The drone landing control device may apply at least one first feature 321 to at least one first recognizer 301 . The drone landing control device may determine whether the target is a landing target based on at least one first output 322 generated by at least one first recognizer 301 .

According to an embodiment, first training features may be generated from training RGB images and training IR images for learning of the at least one first recognizer 301 . The at least one first recognizer 301 has a first training output generated by the at least one first recognizer 301 from first labels corresponding to the training RGB images and training IR images and first training features. can be learned based on A parameter of the at least one first recognizer 301 may be optimized in a direction to minimize a learning error defined based on the difference between the first labels and the first training outputs.

When the target is the landing target, the drone landing control apparatus sets at least one second feature 323 corresponding to the at least one pre-learned second recognizer 302 based on the at least one first output 322. can create

The drone landing control device may apply the at least one second feature 323 to the at least one second recognizer 302 . The drone landing control device may generate the position of the target, which is the landing target, based on at least one second output 324 generated by the at least one second recognizer 302 . Here, the position of the target may be the above-described precise position or the second position.

According to one embodiment, for learning of the at least one second recognizer 302, second training features may be generated from the first training outputs. The at least one second recognizer 302 is configured based on second training outputs generated by the at least one second recognizer 302 from the second training features and second labels corresponding to the first training outputs. can be learned A parameter of the at least one second recognizer 302 may be optimized to minimize a learning error defined based on the difference between the second labels and the second training outputs.

The drone landing control apparatus may generate at least one third feature 325 corresponding to the at least one pre-learned third recognizer 303 based on the at least one second output 324 . According to an embodiment, the drone landing control device may generate at least one third feature 325 corresponding to the pre-learned at least one third recognizer 303 based on the precise location and the location of the drone. .

The drone landing control device may apply at least one third feature 325 to at least one third recognizer 303 . The drone landing control device may generate control information for landing of the drone based on at least one third output 326 generated by at least one third recognizer 303 .

According to an embodiment, third training features may be generated from the second training outputs for learning of the at least one third recognizer 303 . Alternatively, third training features may be generated from the drone's training precise positions and positions. The at least one third recognizer 303 is based on third training outputs generated by the at least one third recognizer 303 from the third training features and third labels corresponding to the second training outputs. can be learned A parameter of the at least one third recognizer 303 may be optimized in a direction to minimize a learning error defined based on the difference between the third labels and the third training outputs. Various techniques can be adopted and applied to the above-described features, methods of defining input and output of recognizers, or learning methods.

4 is a diagram for explaining a drone landing control method according to an embodiment.

4(a) to 4(c), the drone landing control device recognizes a target using various combinations of at least one pre-learned recognizer, detects the location of the target, and provides control information for landing. can create

Referring to FIG. 4 (a), the drone landing control device may use a first recognizer 401 and a second recognizer 402. The drone landing control apparatus may generate a feature 403 based on the RGB image and the IR image of the target and apply the generated feature 403 to the first recognizer 401 . The drone landing control device may determine whether the target is a landing target based on the output 404 of the first recognizer 401 . The drone landing control device may generate a feature based on the output 404 of the first recognizer 401 and apply the generated feature to the second recognizer 402 . The drone landing control device may recognize the location of the target based on the output 405 of the second recognizer 402 . The drone landing control device may generate control information for landing of the drone based on the output 406 of the second recognizer 402 .

Referring to FIG. 4(b), the drone landing control device may use a first recognizer 411 and a second recognizer 412. The drone landing control device may generate a feature 413 based on the RGB image and the IR image of the target and apply the generated feature 413 to the first recognizer 411 . The drone landing control device may determine whether the target is a landing target based on the output 414 of the first recognizer 411 . The drone landing control device may recognize the location of the target based on the output 415 of the first recognizer 411 . The drone landing control device may generate a feature based on the outputs 414 and 415 of the first recognizer 411 and apply the created feature to the second recognizer 412 . The drone landing control device may generate control information for landing of the drone based on the output 416 of the second recognizer 412 .

Referring to FIG. 4(c), the drone landing control device may use the first recognizer 421. According to an embodiment, the first recognizer 421 may be pre-learned in an end-to-end format. The drone landing control device may generate a feature 422 based on the RGB image and the IR image of the target and apply the generated feature 422 to the first recognizer 421 . The drone landing control device may determine whether the target is a landing target based on the output 423 of the first recognizer 421 . The drone landing control device may recognize the location of the target based on the output 424 of the first recognizer 421 . The drone landing control device may generate control information for landing of the drone in the output 425 of the first recognizer 421 .

The aforementioned recognizers can be modified in various aspects, and can be learned by applying various techniques.

5 is a diagram for explaining a learning method according to an exemplary embodiment.

The learning device may train at least one recognizer 501 . The learning device is a device for learning an recognizer for drone landing control, and may be implemented as, for example, a software module, a hardware module, or a combination thereof. The learning device can train the above-mentioned recognizers, and its parameters can be optimized to match the characteristics or inputs and outputs of the above-mentioned recognizers. Although an embodiment of learning is described with reference to FIG. 5 , aspects of learning are not limited thereto, and learning may be performed to match the above-described recognizers.

Referring to FIG. 5 , the learning device generates a plurality of features 504 from a training RGB image 502 and a training IR image 503, and transmits the generated features 504 to a plurality of recognizers 501. can be authorized. The learning device may obtain training outputs 505 generated by recognizers 501 . The learning device may generate a learning error based on a label 506 corresponding to the training RGB image 502 and the training IR image 503 and the training output 505 . The learning device may optimize the parameters of recognizers 501 based on the learning error. For example, the recognizers 501 may be implemented in various ways such as a convolutional neural network (CNN) or a recurrent neural network (RNN), and various techniques such as a gradient descent technique may be employed for parameter optimization. The learning device may repeat the parameter optimization operation until at least one of the learning error, the amount of change in the learning error, and the degree of optimization of the parameters satisfies a predefined condition, and the learning is completed.

6 is a flowchart illustrating a drone landing control method according to an embodiment.

Referring to FIG. 6 , the GCS may receive a precise location of a target generated from a drone (601). The GCS may create an offset based on the reference and precise positions of the target (602). The GCS may determine whether the generated offset meets a predefined range (603). Based on the determination result, the GCS may transmit a landing permission signal to the drone landing control device (604). Here, since the above description may be applied to the operations of the drone landing control device and the GCS, a detailed description thereof will be omitted.

7 is a flowchart illustrating a drone landing control method according to an embodiment.

Referring to FIG. 7, the drone landing control device is based on at least one of information obtained from an ultrasonic sensor and a GPS sensor of a drone and at least one of an RGB image and an IR image of a target obtained from an optical sensor and an IR sensor of a drone. Features can be created (701). The drone landing control device recognizes the target as a landing target for the drone to land from the generated features using a recognizer learned to recognize the target to be landed by the drone, and controls the precise location of the target, which is the landing target, and landing of the drone. At least one of the pieces of information may be generated (702). The drone landing control device may directly generate determination results from features using a pre-learned recognizer in an end-to-end format. Here, the recognizer pre-learned in an end-to-end format can be implemented similarly to the recognizer described above with reference to FIG. 4(c).

8 is an exemplary diagram of a configuration of a device according to an embodiment.

Referring to FIG. 8 , a device 800 includes a processor 801 , a memory 802 , a sensor/camera 803 , a communication interface 804 , an actuator 805 and a data bus 806 . The memory 802 may record applications, programs, or software for processing commands related to drone landing control, and may record commands for executing operations described with reference to FIGS. 1 to 7 . The device 800 may be any one or a combination of the above-described system, drone landing control device, and GCS. The processor 801 may execute a program and control the device 800 . Program codes executed by the processor 801 may be stored in the memory 802 . The device 800 may be connected to an external device (eg, a personal computer or network) via a communication interface 804 and exchange data. The processor 801 may load and execute a program recorded in the memory 802 . Here, since the above-described embodiments are applied to the operations of the processor 801, duplicate descriptions are omitted.

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), programmable logic units (PLUs), microprocessors, or any other device capable of executing and responding to instructions. A processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. You can command the device. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (6)

Recognizing a target 203 and detecting a first position of the target 203 using an ultrasonic sensor and a Global Positioning System (GPS) sensor of the drone 201;
obtaining an RGB image and an IR image of the target 203 using an optical sensor and an IR (Infrared Ray) sensor of the drone 201;
The target 203 is obtained from the RGB images 311 and the IR images 312 using a recognizer learned to recognize the targets 203 on which the drone 201 will land from the RGB images and the IR images. Recognizing the drone 201 as a landing target 203 to land on, and detecting a second position of the target 203, which is the landing target 203;
generating a precise position of the target 203, which is a landing target 203, using the first position and the second position;
Recognizing a direction using a geomagnetic sensor and setting a landing direction;
and
Controlling the landing of the drone 201 based on the precise position of the target 203 as the landing target 203, the landing direction, and the position of the drone 201
including,
Detecting the second position of the target 203
generating at least one feature corresponding to at least one pre-learned recognizer based on the RGB image and the IR image;
determining whether the target (203) is a landing target (203) based on at least one output produced by the at least one recognizer from the at least one feature; and
If the target 203 is the landing target 203, detecting a second position corresponding to at least one of the RGB image and the IR image
containing
Drone 201 landing control method.
According to claim 1,
The step of creating the feature is
generating at least one feature corresponding to the recognizer, further based on information obtained from the ultrasonic sensor and the GPS sensor;
including,
Drone 201 landing control method.
Recognizing a target 203 and detecting a first position of the target 203 using an ultrasonic sensor and a Global Positioning System (GPS) sensor of the drone 201;
obtaining an RGB image and an IR image of the target 203 using an optical sensor and an IR (Infrared Ray) sensor of the drone 201;
The drone 201 identifies the targets 203 from the RGB images and the IR images by using a recognizer learned to recognize the targets 203 on which the drone 201 will land from the RGB images and the IR images. Recognizing a landing target 203 to land on and detecting a second position of the target 203, which is the landing target 203;
generating a precise position of the target 203, which is a landing target 203, using the first position and the second position; and
Controlling the landing of the drone 201 based on the precise position of the target 203 as the landing target 203 and the position of the drone 201
Including, The step of detecting the second position of the target 203
generating at least one first feature 321 corresponding to at least one pre-learned first recognizer 301 based on the RGB image and the IR image;
based on at least one first output (322) produced by the first recognizer (301) from the first feature (321), determining whether the target (203) is a landing target (203);
When the target 203 is the landing target 203, at least one second feature 323 corresponding to the pre-learned at least one second recognizer 302 is generated based on the first output 322 doing; and
based on at least one second output (324) produced by the second recognizer (302) from the second feature (323), generating a second location of the target (203) that is a landing target (203);
including,
For learning of the first recognizer 301, first training features are generated from training RGB images and training IR images,
The first recognizer 301 based on first labels corresponding to the training RGB images and the training IR images and first training outputs generated by the first recognizer from the first training features being learned,
For learning of the second recognizer (302), second training features are generated from the first training outputs;
The second recognizer 302 is learned based on second training outputs generated by the second recognizer 302 from second labels corresponding to the second training outputs and the second training features. ,
Drone 201 landing control method.
Recognizing a target 203 and detecting a first position of the target 203 using an ultrasonic sensor and a Global Positioning System (GPS) sensor of the drone 201;
obtaining an RGB image and an IR image of the target 203 using an optical sensor and an IR (Infrared Ray) sensor of the drone 201;
The drone 201 identifies the targets 203 from the RGB images and the IR images by using a recognizer learned to recognize the targets 203 on which the drone 201 will land from the RGB images and the IR images. Recognizing a landing target 203 to land on and detecting a second position of the target 203, which is the landing target 203;
generating a precise position of the target 203, which is a landing target 203, using the first position and the second position; and
Controlling the landing of the drone 201 based on the precise position of the target 203 as the landing target 203 and the position of the drone 201
Including, The step of controlling the landing of the drone 201
generating at least one third feature (325) corresponding to at least one pre-learned third recognizer (303) based on the precise position and the position of the drone (201); and
generating control information for landing of the drone 201 based on at least one third output 326 generated by the third recognizer 303 from the third feature 325;
including,
For learning of the third recognizer (303), third training features are generated from training precision positions and positions of the drone (201);
The third recognizer 303 is based on the third training outputs generated by the third recognizer 303 from the training precise locations and third labels corresponding to the locations and the third training features. learned by doing
Drone 201 landing control method.
According to claim 1,
The step of creating the precise position is
based on the first location and the second location, generating at least one feature corresponding to at least one recognizer learned to recognize precise locations of landing targets (203); and
generating a precise position of the target (203) based on at least one output the recognizer produces from the feature;
including,
Drone 201 landing control method.
Recognizing a target 203 and detecting a first position of the target 203 using an ultrasonic sensor and a Global Positioning System (GPS) sensor of the drone 201;
obtaining an RGB image and an IR image of the target 203 using an optical sensor and an IR (Infrared Ray) sensor of the drone 201;
The drone 201 identifies the targets 203 from the RGB images and the IR images by using a recognizer learned to recognize the targets 203 on which the drone 201 will land from the RGB images and the IR images. Recognizing a landing target 203 to land on and detecting a second position of the target 203, which is the landing target 203;
generating a precise position of the target 203, which is a landing target 203, using the first position and the second position;
controlling the landing of the drone 201 based on the precise position of the target 203 as the landing target 203 and the position of the drone 201;
transmitting the generated precise location to a Ground Control Station (GCS) 204 of the target 203;
receiving an offset generated by the GCS (204) based on the reference position and the precise position of the target (203);
receiving a landing authorization signal generated by the GCS (204) in response when the offset meets a predefined range; and
Based on the offset, correcting the precise position.
including,
The step of controlling the landing of the drone 201 is
Generating control information for landing of the drone 201 based on the corrected precise position and the position of the drone 201
including,
Drone 201 landing control method.

Images