KR102486768B1 - Unmanned drone for automatically setting moving path according to detection situation, and operating method thereof - Google Patents

Abstract

Unmanned aerial vehicle starts. The present unmanned aerial vehicle includes a camera, a drive control unit for controlling flight of the unmanned aerial vehicle, a communication unit for communicating with at least one control server, and a processor. The processor controls the flight of the unmanned aerial vehicle along a preset first path through a driving control unit, detects at least one object based on an image acquired through a camera during the flight of the unmanned aerial vehicle, and when the object is detected , According to the second path for tracking the detected object, the flight of the unmanned aerial vehicle is controlled through the driving control unit, and the location information of the unmanned aerial vehicle and information on the second path are transmitted to the control server through the communication unit.

Description

An unmanned aerial vehicle that automatically sets a movement path according to detection situations, and an operation method { UNMANNED DRONE FOR AUTOMATICALLY SETTING MOVING PATH ACCORDING TO DETECTION SITUATION, AND OPERATING METHOD THEREOF }

The present disclosure relates to an unmanned aerial vehicle for detection, and more particularly, to an unmanned aerial vehicle that changes a flight path based on an object detection situation.

Conventionally, a system for searching for a missing person by operating an unmanned aerial vehicle after an accident has been used, but there is not much development of a system that detects and takes first responders by region using unmanned aerial vehicles operating in each major base or in each major region. situation is not.

In particular, a remote communication and control technology of an unmanned aerial vehicle that shares a situation with one or more control servers according to object detection needs to be built in more detail.

Registered Patent Publication No. 10-21868230000 (drone control system)

The present disclosure provides an unmanned aerial vehicle that changes a route according to detection conditions, communicates with one or more control servers, and shares detection conditions.

The objects of the present disclosure are not limited to the above-mentioned objects, and other objects and advantages of the present disclosure not mentioned above can be understood by the following description and will be more clearly understood by the embodiments of the present disclosure. Further, it will be readily apparent that the objects and advantages of the present disclosure may be realized by means of the instrumentalities and combinations indicated in the claims.

An unmanned aerial vehicle according to an embodiment of the present disclosure includes a camera, a driving control unit for controlling the flight of the unmanned aerial vehicle, a communication unit for communicating with at least one control server, and connected to the camera, the driving control unit, and the communication unit. contains the processor. The processor controls the flight of the unmanned aerial vehicle along a first preset path through the driving control unit, detects at least one object based on an image obtained through the camera while the unmanned aerial vehicle is in flight, and , When the object is detected, the flight of the unmanned aerial vehicle is controlled through the drive control unit according to the second path for tracking the detected object, and location information of the unmanned aerial vehicle and information on the second path are provided. It is transmitted to the control server through the communication unit.

When the object is detected, the processor identifies the location of the rescue movable object by communicating with at least one rescue movable object through the communication unit, and when the rescue movable object is located within a preset distance from the detected object, The flight of the unmanned aerial vehicle may be controlled through the driving control unit according to the third path for turning around the sensed object.

In addition, the processor identifies a control server closest to the location of the unmanned aerial vehicle among a plurality of control servers provided for each area, and transmits the location information, information on the second path, and the image of the sensed object to the control server. It can be transmitted to the identified control server through the communication unit.

Here, the processor identifies the location of the sensed object in real time, obtains images of the sensed object in real time, when the unmanned aerial vehicle is out of a communication area with the identified control server, and the unmanned aerial vehicle Controls the drive control unit to move to an area in which communication with the identified control server is possible, and when the unmanned aerial vehicle arrives in an area in which communication with the identified control server is possible, information on the location identified in real time and the real time The acquired images may be transmitted to the control server.

At this time, the processor, after the information on the location identified in real time and the images acquired in real time are transmitted to the control server, the unattended location of the object predicted based on the location identified in real time. The driving control unit may be controlled to move the aircraft.

In this case, the processor may determine the expected location of the object by inputting the identified location in real time to at least one artificial intelligence model trained to predict a moving path.

Meanwhile, the processor controls the driving control unit to move the unmanned aerial vehicle to at least one charging point when the remaining battery level of the unmanned aerial vehicle is less than a first threshold while the unmanned aerial vehicle is flying along the first path, and , When the remaining battery level of the unmanned aerial vehicle is less than a second threshold smaller than the first threshold while the unmanned aerial vehicle is flying along the second path as the object is detected, the unmanned aerial vehicle moves to at least one charging point The driving control unit may be controlled to do so.

An operation method of an unmanned aerial vehicle according to an embodiment of the present disclosure includes the steps of moving along a first preset path, and detecting at least one object based on an image obtained through a camera while the unmanned aerial vehicle is moving. , When the object is detected, moving along a second path for tracking the detected object, and transmitting location information of the unmanned aerial vehicle and information on the second path to at least one control server. do.

An unmanned aerial vehicle and an operation method according to the present disclosure have an advantage of performing flexible situation sharing by automatically changing a moving path according to an object detection situation.

The unmanned aerial vehicle and operation method according to the present disclosure have an effect of increasing the accuracy and efficiency of detection through an algorithm comprehensively reflecting communication conditions, viewing conditions, batteries, and the like.

1 is a diagram for explaining an operation in which an unmanned aerial vehicle communicates with at least one control server and detects a path according to an embodiment of the present disclosure;
2 is a block diagram for explaining the configuration of an unmanned aerial vehicle according to an embodiment of the present disclosure;
3 is a flowchart for explaining the operation (operation method) of an unmanned aerial vehicle according to an embodiment of the present disclosure;
4 is a diagram for explaining an operation in which an unmanned aerial vehicle performs a turning flight based on a detected object according to an embodiment of the present disclosure;
5 is a diagram for explaining an operation in which an unmanned aerial vehicle sequentially performs information collection and situation sharing according to a communication distance with a control server according to an embodiment of the present disclosure; and
6 is a block diagram for explaining the configuration of an unmanned aerial vehicle according to various embodiments of the present disclosure.

Prior to a detailed description of the present disclosure, the method of describing the present specification and drawings will be described.

First, terms used in the present specification and claims are general terms in consideration of functions in various embodiments of the present disclosure. However, these terms may vary depending on the intention of a technician working in the art, legal or technical interpretation, and the emergence of new technologies. In addition, some terms are arbitrarily selected by the applicant. These terms may be interpreted as the meanings defined in this specification, and if there is no specific term definition, they may be interpreted based on the overall content of this specification and common technical knowledge in the art.

In addition, the same reference numerals or numerals in each drawing attached to this specification indicate parts or components that perform substantially the same function. For convenience of description and understanding, the same reference numerals or symbols are used in different embodiments. That is, even if all components having the same reference numerals are shown in a plurality of drawings, the plurality of drawings do not mean one embodiment.

Also, in the present specification and claims, terms including ordinal numbers such as “first” and “second” may be used to distinguish between elements. These ordinal numbers are used to distinguish the same or similar components from each other, and the meaning of the term should not be construed as being limited due to the use of these ordinal numbers. For example, the order of use or arrangement of elements associated with such ordinal numbers should not be limited by the number. If necessary, each ordinal number may be used interchangeably.

In this specification, singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "comprise" or "consist of" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other It should be understood that the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof is not precluded.

In the embodiments of the present disclosure, terms such as “module,” “unit,” and “part” are terms used to refer to components that perform at least one function or operation, and these components are hardware or software. It may be implemented or implemented as a combination of hardware and software. In addition, a plurality of "modules", "units", "parts", etc. are integrated into at least one module or chip, except for cases where each of them needs to be implemented with separate specific hardware, so that at least one processor can be implemented as

Also, in an embodiment of the present disclosure, when a part is said to be connected to another part, this includes not only a direct connection but also an indirect connection through another medium. In addition, the meaning that a certain part includes a certain component means that it may further include other components without excluding other components unless otherwise stated.

1 is a diagram for explaining an operation in which an unmanned aerial vehicle performs communication with at least one control server and detects a path according to an embodiment of the present disclosure.

Referring to FIG. 1 , the unmanned aerial vehicle 100 may perform communication with at least one of a plurality of control servers 200-1, 2, ... provided for each area.

Specifically, the unmanned aerial vehicle 100 may identify the location of the unmanned aerial vehicle 100 using various sensors such as GPS (Global Positioning System), and transmit the location information of the unmanned aerial vehicle 100 to the control server 200. can

In addition, the unmanned aerial vehicle 100 may set a flight path or flight time based on a control signal received from the control server 200 .

The unmanned aerial vehicle 100 may fly and detect various routes, such as at least one coastal area, a dangerous area, and various other reconnaissance or detection areas.

Meanwhile, referring to FIG. 1, the unmanned aerial vehicle 100 uses a control server 200-1 closest to the current location of the unmanned aerial vehicle 100 among a plurality of control servers 200-1, 2, ... provided for each region. can communicate with

Alternatively, the unmanned aerial vehicle 100 may perform communication with the control server 200-1 closest to the route 11 on which the unmanned aerial vehicle 100 is currently flying.

Alternatively, the unmanned aerial vehicle 100 is set to manage the area including the current location or the current flight path of the unmanned aerial vehicle 100 among a plurality of control servers 200-1, 2, ... that manage different areas. Communication with the control server 200-1 may also be performed.

The unmanned aerial vehicle 100 may transmit information about an image of an object detected during detection, the location of the object, and the location/route of the unmanned aerial vehicle 100 to the control server 200-1.

Referring to FIG. 1 , when a specific object 12 such as a missing person or a person in distress is detected during detection of a path 11, the unmanned aerial vehicle 100 flies along a path for tracking the object 12 and locates the object. can be tracked and monitored.

* In addition, the unmanned aerial vehicle 100 may transmit an image of an object and location information of the object to the control server 200-1.

As such, since the unmanned aerial vehicle 100 according to the present disclosure automatically changes a path and shares the situation according to the object detection situation, it is possible to transmit meaningful data to the control server by performing active detection.

Through the following drawings, the configuration and operation of the unmanned aerial vehicle 100 according to the present disclosure will be described in more detail.

2 is a block diagram for explaining the configuration of an unmanned aerial vehicle according to an embodiment of the present disclosure.

Referring to FIG. 2 , the unmanned aerial vehicle 100 may include a camera 110, a communication unit 120, a driving control unit 130, a processor 140, and the like.

The camera 110 is a component for obtaining various images of the external environment.

The camera 110 may include various types of cameras such as an RGB camera, a Time of Flight (TOF) camera, a thermal camera, and a stereo camera.

The camera 110 may include cameras having various viewing angles, or may include a plurality of cameras.

Meanwhile, although not shown, the unmanned aerial vehicle 100 may include at least one light output unit for assisting the camera 110 in shooting. The light output unit may include at least one LED (Light Emitting Diode) or other laser output means, but is not limited thereto.

In this case, the processor 140 may control the light output unit to output light to a point where the camera 110 captures the image when the camera 110 captures the image, and as a result, detection using the image of the camera 110 is possible even at night. It works if it is possible.

The communication unit 120 is a configuration for the unmanned aerial vehicle 100 to transmit and receive data with various external devices, and may include at least one circuit for communication.

Specifically, the communication unit 120 may communicate with the control servers 200-1, 2, ..., rescue team servers, rescue boats, flying vehicles, rescue vehicles, other unmanned aerial vehicles, and other various terminals.

The communication unit 120 includes Transmission Control Protocol/Internet Protocol (TCP/IP), User Datagram Protocol (UDP), Hyper Text Transfer Protocol (HTTP), Secure Hyper Text Transfer Protocol (HTTPS), File Transfer Protocol (FTP), and SFTP ( Various types of information may be transmitted and received with one or more external electronic devices using communication protocols such as Secure File Transfer Protocol (MQTT) and Message Queuing Telemetry Transport (MQTT).

To this end, the communication unit 120 may be connected to an external device based on a network implemented through wireless communication. At this time, the communication unit 120 may be directly connected to an external device, but may also be connected to an external electronic device through one or more external servers (eg, Internet Service Provider (ISP)) providing a network.

The network may be a Personal Area Network (PAN), a Local Area Network (LAN), a Wide Area Network (WAN), etc., depending on the area or size, and an intranet, It may be an extranet or the Internet.

Wireless communication includes LTE (long-term evolution), LTE-A (LTE Advance), 5G (5th generation) mobile communication, CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), WiBro (Wireless Broadband), GSM (Global System for Mobile Communications), DMA (Time Division Multiple Access), WiFi (Wi-Fi), WiFi Direct, Bluetooth, NFC (near field communication), Zigbee, etc. can include

Here, the communication unit 120 may include a network interface or network chip according to the above-described wireless communication method. On the other hand, the communication method is not limited to the above-described example, and may include a newly appearing communication method according to the development of technology.

The driving control unit 130 is a component for controlling the flight of the unmanned aerial vehicle 100. The driving control unit 130 controls take-off and landing of the unmanned aerial vehicle 100, movement of the unmanned aerial vehicle 100 in a three-dimensional direction, rotation of the unmanned aerial vehicle 100 in pitch, roll, and yaw directions, etc. can control.

For example, the drive controller 130 may include at least one control circuit for controlling a rotation motor for driving rotation of each propeller provided in the unmanned aerial vehicle 100 . In addition, the driving controller 130 may drive at least one engine (eg, a jet engine) provided in the unmanned aerial vehicle 100 .

The processor 140 is a component for controlling the overall configuration included in the unmanned aerial vehicle 100, a central processing unit (CPU), an application processor (AP), a graphic processing unit (GPU), a video processing unit (VPU), and an NPU. (Neural Processing Unit).

For example, the processor 140 may control each component of the unmanned aerial vehicle 100 by executing instructions stored in the memory of the unmanned aerial vehicle 100 .

The operation of the unmanned aerial vehicle 100 according to the control of the processor 140 according to various embodiments of the present disclosure will be described through the following drawings.

3 is a flowchart for explaining an operation of an unmanned aerial vehicle according to an embodiment of the present disclosure.

Referring to FIG. 3 , the unmanned aerial vehicle 100 may move along a preset first path (S310).

Here, the preset first route may correspond to a route for scouting or detecting various areas such as a coastal area, a maritime area, a mountainous area, and a dangerous area.

The preset first route may be set through at least one control server.

For example, the control server may select at least one area in which accidents frequently occur according to the history of accidents for each of a plurality of areas included on the map, and set the corresponding area as a route for the flight of the unmanned aerial vehicle 100.

As another example, the control server may receive a user input for setting at least one flight path on the map, and may transmit the flight path set according to the user input to the unmanned aerial vehicle 100 . Here, the user input may be received through a user input unit provided in the control server or through at least one terminal device capable of communicating with the control server 100 .

As another example, the control server may set a flight path based on an accident point. In this case, the control server may set a new route for patrolling an area within a certain distance around the accident occurrence point.

Alternatively, the unmanned aerial vehicle 100 may directly set a route based on the information on the accident occurrence point received from the control server.

The processor 140 may identify location information of the unmanned aerial vehicle 100 through a GPS sensor or the like, and may transmit real-time movement path and real-time location information to the control server through the communication unit 120. At this time, the processor 140 may perform communication with a control server closest to the unmanned aerial vehicle 100 among a plurality of control servers provided for each region.

Then, the unmanned aerial vehicle 100 may detect at least one object based on the image obtained through the camera 110 during flight of the first route (S320).

As an example, the processor 140 may identify whether an object having a predetermined shape or a predetermined color exists in an image obtained through the camera 110 .

For example, the processor 140 may analyze the image to identify whether an object having a predetermined shape (eg, a human shape, an animal shape, a ship shape, an aircraft shape, etc.) exists in the image.

Here, the processor 140 may use at least one module or artificial intelligence model for identifying an object having a preset shape. Specifically, the processor 140 may input an image captured through the camera 110 to the above-described artificial intelligence model to detect the presence or absence of an object having a preset shape.

This artificial intelligence model may be a model trained to identify a corresponding shape based on images including various objects (eg, people, animals, ships, aircraft, etc.) having preset shapes. Here, each image may be used for training while the outline or shape of the object included in each image is marked, but is not limited thereto.

Also, the processor 140 may acquire a depth image through a depth camera and identify a shape of each object included in the obtained depth image. In this case, the processor 140 may determine whether an object having a preset shape exists.

In addition, the processor 140 may detect whether an object having a preset color exists based on R/G/B values for each pixel of an image captured through an RGB camera.

For example, information on a predetermined color (ex. red, white, etc.) that is difficult to exist in a natural terrain without human traces may be pre-stored in the memory of the unmanned aerial vehicle 100 .

In this case, the processor 140 identifies whether the R/G/B value of at least one pixel in the image matches a preset color, and if there are more than a threshold number of pixels matching the preset color in the image, It may be determined that at least one object exists.

Also, as an embodiment, the processor 140 may input an image to an artificial intelligence model trained to identify at least one predetermined object, and recognize at least one object based on an output of the artificial intelligence model.

This artificial intelligence model may be a model trained to identify objects based on a plurality of images including predetermined objects (eg, people, animals, ships, aircraft, etc.). For example, this artificial intelligence model is CNN (Convolutional Neural Network) and can act as a classifier that outputs the type of object included in the image.

Also, as an example, the processor 140 may detect at least one object based on a thermal image captured by a thermal camera. The thermal camera may be, for example, an infrared-based camera, but is not limited thereto. Through the thermal camera, the unmanned aerial vehicle 100 can detect various objects even at night.

As a specific example, the processor 140 may detect at least one object having a predetermined temperature range that matches the body temperature of a human or animal within a thermal image.

In this case, the processor 140 may acquire an RGB image by capturing an area including an object detected through a thermal image with an RGB camera. Also, the processor 140 may recognize an object by analyzing an image captured through an RGB camera.

As such, when an object is detected according to various embodiments described above, the unmanned aerial vehicle 100 may move along the second path for tracking the detected object (S330).

Here, the second path may correspond to a path for tracking an object according to the movement of the detected object or a path for turning around the detected object.

For example, the processor 140 obtains a plurality of images by controlling the camera 110 to capture an image of an object in real time, and determines a moving direction of the object based on a positional change of the object included in the obtained plurality of images. can be identified.

In this case, the processor 140 may control the driving controller 130 to move the unmanned aerial vehicle 100 according to the moving direction of the identified object.

As another example, the processor 140 may control the drive control unit 130 by setting a path for turning an area within a predetermined distance based on the position of the detected object.

Meanwhile, the unmanned aerial vehicle 100 may fly on a second path for tracking the detected object, and then fly on a third path for turning around the object when at least one rescue vehicle approaches. In this regard, a specific embodiment will be described later with reference to FIG. 4 .

In addition, the processor 140 may transmit location information of the unmanned aerial vehicle 100 and information on the second path to the control server (S340). At this time, the processor 140 may also transmit information on the sensed or recognized object.

As a result, the control server can identify the location (discovery location) of the object according to the location information of the unmanned aerial vehicle 100, and can identify the changed second path of the unmanned aerial vehicle 100. In addition, the control server may identify the moving state and/or moving direction of the object based on the second path of the unmanned aerial vehicle 100 tracking the object.

Meanwhile, FIG. 4 is a diagram for explaining an operation in which an unmanned aerial vehicle performs a turning flight based on a detected object according to an embodiment of the present disclosure.

Referring to FIG. 4 , the processor 140 may communicate with the movable object 400 for rescue through the communication unit 120 to identify the location of the movable object 400 for rescue.

In this case, the processor 140 may transmit a rescue request to the rescue movable body 400 through the communication unit 120, and may also transmit object location information. At this time, the processor 140 may perform direct communication with the mobile object 400 for rescue through the communication unit 120 or may transmit a rescue request through the control server.

The rescue vehicle 400 may correspond to a rescue ship, a rescue aircraft, a rescue vehicle, and the like, and may include at least one module for communicating with the unmanned aerial vehicle 100 .

And, when the rescue movable body 400 is located within a preset distance from the detected object 11, the processor 140 operates the driving control unit 130 according to the third path 410 for turning around the detected object. ), the flight of the unmanned aerial vehicle 100 may be controlled.

That is, as the object is detected, the unmanned aerial vehicle 100, which has been flying along the second path tracking the object, can fly along the third path 410, which orbits around the object as the rescue vehicle 400 approaches, As a result, there is an advantage in that the sensor of the rescue vehicle 400 or a rescue team member aboard the rescue vehicle 400 can quickly find the location of the object based on the turning flight of the unmanned aerial vehicle 100.

Here, the unmanned aerial vehicle 100 may turn in a circle around a point higher than the altitude of the unmanned aerial vehicle 100 in a vertical direction from the point where the detected object is located. Alternatively, the unmanned aerial vehicle 100 makes a circle around a point higher than the altitude of the unmanned aerial vehicle 100 in the vertical direction from a point moved by a certain distance in the direction of the rescue movable body 400 from the point where the detected object is located. You can draw and turn.

Here, the radius of the circle drawn while turning by the unmanned aerial vehicle 100 may be set differently according to the viewing condition.

For example, the visibility state may be divided into a plurality of stages according to the degree of fog or the concentration of fine dust. Here, it may be defined that the higher the level of the visual field state, the easier it is to secure the peripheral vision. That is, the clearer the fog is, the lower the concentration of fine dust is, the higher the visual field level.

In this case, the processor 140 may identify a visual field state based on a surrounding image captured through the camera 110 .

Here, the processor 140 analyzes each pixel value included in the image to identify noise (eg, fog, fine dust) included in the image, and determines the level of the visual field state to be low as the amount of noise increases.

Alternatively, the processor 140 may use at least one artificial intelligence model trained to determine the visual field state. This artificial intelligence model captures images corresponding to the viewing conditions for each stage (ex. images on sunny days, cloudy days, foggy days, high fine dust images, etc.) It may be a model trained to determine the visual field state based on

Also, the processor 140 may set a path of the turning flight of the unmanned aerial vehicle 100 based on the stage of the determined visual state. Specifically, as the level of the visibility state is lower (the worse the visibility state is), the processor 140 may control the unmanned aerial vehicle 100 to turn while drawing a circle having a larger radius.

In this case, the possibility that the rescue movable body 400 finds the unmanned aerial vehicle 100 increases in weather conditions with poor visibility. On the other hand, in the case of a clear day where the unmanned aerial vehicle 100 can be easily found, as the radius of the path in which the unmanned aerial vehicle 100 turns around an object becomes narrower, the rescue moving body 400 that finds the unmanned aerial vehicle 100 has the advantage of being able to quickly identify the location of an object.

Subsequently, when information indicating that an object has been found is received from the rescue vehicle 400, the processor 140 allows the drone 100 to fly along at least one detection path (eg, a first path) for detecting another object. can control.

Meanwhile, FIG. 5 is a diagram for explaining an operation of sequentially performing information collection and situation sharing according to whether an unmanned aerial vehicle can communicate with a control server according to an embodiment of the present disclosure.

Referring to FIG. 5, the unmanned aerial vehicle 100 communicates with the control server 200-1 closest to the location of the unmanned aerial vehicle 100 among a plurality of control servers 200-1, 2, ... provided for each region. can be done

However, at least a part of the detection path of the unmanned aerial vehicle 100 may deviate from an area where communication is possible between the unmanned aerial vehicle 100 and the control server 200-1.

Whether or not the communication is possible may be determined in real time through the unmanned aerial vehicle 100 .

Specifically, the unmanned aerial vehicle 100 transmits information on the location and route of the unmanned aerial vehicle 100 to the control server 200-1 in real time, and at least one response signal or A control signal may be received.

Here, if the transmission speed of the data / signal transmitted and received with the control server 200-1 is less than the critical speed, or if the success rate of data / signal transmission and reception is less than the threshold value, or if the reception sensitivity of the data / signal is less than the threshold sensitivity, The processor 140 may identify that the unmanned aerial vehicle 100 is out of a communication area with the control server 200-1.

As such, the unmanned aerial vehicle 100 detecting a path out of a communication area with the control server 200-1 may detect at least one object as shown in FIG. 5 (S510). Alternatively, the unmanned aerial vehicle 100 may detect an object within a communicable area and then leave the communicable area while tracking the object.

In this case, the unmanned aerial vehicle 100 may identify the location of the object in real time through at least one sensor (eg, a GPS sensor), and the image of the detected object may also be captured and stored in real time through the camera 110. Yes (S520).

Then, the unmanned aerial vehicle 100 may move to a communication area again (S530).

Specifically, when the storage amount of object images exceeds a certain capacity or the number of stored object images exceeds a certain number, the processor 140 causes the drone 100 to stop tracking the object and move to a communicable area. The driving control unit 130 may be controlled.

Here, the unmanned aerial vehicle 100 may determine whether communication with the control server is possible in real time while moving in the direction where the control server is located.

Thereafter, the unmanned aerial vehicle 100 reaching the communicable area may transmit information on the previously identified location in real time and images acquired (stored) in real time to the control server 200-1 (S540).

Then, the unmanned aerial vehicle 100 may move back to the expected location of the object (S550).

The predicted location may be predicted based on the location of the previously identified object in real time.

For example, the processor 140 of the unmanned aerial vehicle 100 inputs the identified location (by time) in real time prior to at least one artificial intelligence model trained to predict the moving path to determine the expected location of the object. can

In addition, the processor 140 may input information about the position of the object by time and the terrain where the object is located to the artificial intelligence model. The information on topography may include information on whether each point is on the water or on the ground, and information on the water depth or altitude of each point.

In this case, the artificial intelligence model may predict the location of the object in the future based on the location of the object in the terrain by time.

This artificial intelligence model may be a model trained based on information on the position of at least one moving object by time (eg, a record of the position of an actual survivor in the past by time), and may be a Recurrent Neural Network (RNN) model. may, but is not limited thereto.

In addition, this artificial intelligence model may be a model trained based on information on the terrain where the corresponding object was located (ex. information on the terrain where the actual survivor was in distress in the past) in addition to information on the location of the object by time. .

As another example, the expected location of the object may be determined through the control server 200 receiving information on the real-time location of the object from the unmanned aerial vehicle 100 . In this case, the control server 200 may transmit a control signal to move the unmanned aerial vehicle 100 to an expected position.

The unmanned aerial vehicle 100, which has moved to the expected location according to at least one of the above embodiments, may resume detection of the object as it finds the object again.

In this case, the unmanned aerial vehicle 100 may delete the real-time location information and images of the object already transmitted to the control server 200-1 from the memory. As a result, as tracking of the object is resumed, information on the real-time location of the newly collected object and the image of the object may be stored in the memory without a capacity problem.

Meanwhile, the unmanned aerial vehicle 100 may move to at least one charging point according to the remaining battery level. In this regard, information on the location of at least one charging point may be stored in the memory of the unmanned aerial vehicle 100, and the unmanned aerial vehicle 100 may move to the charging point closest to the unmanned aerial vehicle 100 according to the remaining battery level. there is.

Specifically, the processor 140 may control the driving control unit 130 to move the unmanned aerial vehicle 100 to the nearest charging point when the remaining battery capacity of the unmanned aerial vehicle 100 in flight is less than a threshold value.

In this case, the processor 140 may calculate the threshold in real time according to the distance between the nearest charging point and the unmanned aerial vehicle 100 . As a specific example, the processor 140 identifies a value obtained by multiplying the battery usage per unit distance movement of the unmanned aerial vehicle 100 by the distance to the charging point, and sets the threshold of the battery to match the energy amount obtained by multiplying the identified value by 120%. It can be set, but is not limited thereto.

In addition, the processor 140 may differently set a threshold ratio of battery related to a point in time of moving to a charging point, depending on whether an object is detected.

For example, when the remaining battery level of the unmanned aerial vehicle 100 is less than a first threshold value while the unmanned aerial vehicle 100 is flying along a first path that is a detection path, the processor 140 moves the unmanned aerial vehicle 100 to a charging point. It is possible to control the driving control unit 130 to do so.

On the other hand, when the unmanned aerial vehicle 100 is flying along the second path (object tracking) as an object is detected during flight along the first path, the remaining battery capacity of the unmanned aerial vehicle 100 is a second threshold value smaller than the first threshold value. On the premise that it is below the charging point, the processor 140 may control the drive control unit 130 to move the unmanned aerial vehicle 100 to the charging point.

That is, when an object is detected and is in tracking flight, movement to a charging point may be relatively delayed compared to a case in which an object is not detected and is in general detection flight. As a result, tracking of the sensed object may proceed more smoothly.

However, even if the second threshold is smaller than the first threshold, since the unmanned aerial vehicle 100 must not be discharged without reaching the charging point, the second threshold is charged according to the battery usage per unit distance of the unmanned aerial vehicle 100. It should be set to be greater than the amount of energy multiplied by the distance to the point.

Meanwhile, FIG. 6 is a block diagram for explaining the configuration of an unmanned aerial vehicle according to various embodiments of the present disclosure.

The unmanned aerial vehicle 100 may further include a sensor unit 150, a memory 160, and the like in addition to the camera 110, the communication unit 120, the drive control unit 130, and the processor 140.

The sensor unit 150 may include various types of sensors for obtaining information about the surrounding environment. Specifically, the sensor unit 150 may include a GPS sensor, a geomagnetic sensor, an acceleration sensor, a wind direction/speed sensor, a LIDAR sensor, and the like.

The processor 140 may control the driving control unit 130 to drive the flight of the unmanned aerial vehicle 100 based on the sensing data obtained through the various sensors described above.

Specifically, the processor 140 may identify and adjust the position, speed, acceleration, direction, etc. of the unmanned aerial vehicle 100 through the sensor unit 150 in real time.

The processor 140 may acquire an image through the camera 110 when at least one object is sensed based on the sensing data. For example, when at least one object is sensed through sensing data obtained through a lidar sensor, the processor 140 may control the camera 110 to photograph the sensed object. In this case, the processor 140 may recognize an object by analyzing an image obtained through the camera 110 .

The memory 160 is a configuration for storing an operating system (OS) for controlling the overall operation of components in the unmanned aerial vehicle 100, at least one instruction, and data.

The memory 160 may include non-volatile memory such as ROM and flash memory, and may include volatile memory composed of DRAM and the like. Also, the memory 160 may include a hard disk, a solid state drive (SSD), and the like.

The memory 160 may store artificial intelligence models according to various embodiments described above. The memory 160 includes a Convolutional Neural Network (CNN), a Deep Neural Network (DNN), a Recurrent Neural Network (RNN), a Restricted Boltzmann Machine (RBM), a Deep Belief Network (DBN), a Bidirectional Recurrent Deep Neural Network (BRDNN), a GAN It may include various artificial intelligence models such as (Generative Adversarial Network) and Deep Q-Networks, but is not limited thereto.

The memory 160 may store an image of an object captured by the camera 110, information on the real-time location of the object, information on the real-time location of the unmanned aerial vehicle 100, information on at least one preset route, and the like. there is.

Meanwhile, the various embodiments described above may be implemented by combining a plurality of embodiments as long as they do not conflict with each other.

Meanwhile, various embodiments described above may be implemented in a recording medium readable by a computer or a similar device using software, hardware, or a combination thereof.

According to the hardware implementation, the embodiments described in this disclosure are application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), and field programmable gate arrays (FPGAs). ), processors, controllers, micro-controllers, microprocessors, and electrical units for performing other functions.

In some cases, the embodiments described herein may be implemented by a processor itself. According to software implementation, embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules described above may perform one or more functions and operations described herein.

On the other hand, computer instructions or computer programs for performing processing operations in each device according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. can Computer instructions or computer programs stored in such a non-transitory computer readable medium, when executed by a processor of a specific device such as an unmanned aerial vehicle, cause the specific device to perform the operation of the unmanned aerial vehicle according to various embodiments described above.

A non-transitory computer readable medium is a medium that stores data semi-permanently and is readable by a device, not a medium that stores data for a short moment, such as a register, cache, or memory. Specific examples of the non-transitory computer readable media may include CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

Although the preferred embodiments of the present disclosure have been shown and described above, the present disclosure is not limited to the specific embodiments described above, and is common in the technical field belonging to the present disclosure without departing from the gist of the present disclosure claimed in the claims. Of course, various modifications and implementations are possible by those with knowledge of, and these modifications should not be individually understood from the technical spirit or perspective of the present disclosure.

100: drone 110: camera
120: communication unit 130: driving control unit
140: processor

Claims (2)

In unmanned aerial vehicles,
camera;
a driving control unit controlling flight of the unmanned aerial vehicle;
a communication unit for communicating with at least one control server; and
Including; a processor connected to the camera, the driving control unit, and the communication unit;
the processor,
Controlling the flight of the unmanned aerial vehicle along a preset first path through the driving control unit;
Detecting at least one object based on an image obtained through the camera during flight of the unmanned aerial vehicle;
When the object is detected, the flight of the unmanned aerial vehicle is controlled through the driving control unit according to a second path for tracking the detected object;
Transmitting location information of the unmanned aerial vehicle and information on the second route to the control server through the communication unit;
the processor,
Controlling the driving control unit to move the unmanned aerial vehicle to at least one charging point when the remaining battery capacity of the unmanned aerial vehicle is less than a first threshold while the unmanned aerial vehicle is flying along the first route;
When the remaining battery level of the unmanned aerial vehicle is less than a second threshold smaller than the first threshold while the unmanned aerial vehicle is flying along the second route as the object is detected, the unmanned aerial vehicle moves to at least one charging point. An unmanned aerial vehicle that controls the driving control unit. In the operation method of the unmanned aerial vehicle,
moving according to a first preset path;
detecting at least one object based on an image acquired through a camera while the unmanned aerial vehicle is moving;
moving along a second path for tracking the detected object when the object is detected; and
Transmitting the location information of the unmanned aerial vehicle and information about the second path to at least one control server;
The operation method of the unmanned aerial vehicle,
moving to at least one charging point when the remaining battery capacity of the unmanned aerial vehicle is less than a first threshold while the unmanned aerial vehicle is flying along the first path; and
moving to the at least one charging point when the remaining battery capacity of the unmanned aerial vehicle is less than a second threshold smaller than the first threshold while the unmanned aerial vehicle is flying along the second route as the object is detected; Further comprising, a method of operating an unmanned aerial vehicle.

Images