KR102132945B1 - UAV landing induction method - Google Patents

Abstract

A method for inducing unmanned landing is disclosed. According to an aspect of the present invention, the server receiving the eleventh information including the azimuth information of the wireless terminal from the wireless terminal; Receiving, by the server, twelfth information including angle information inclined with respect to the surface of the wireless terminal from the wireless terminal; Generating, by the server, 21st information including information about the landing point by calculating a landing point of the unmanned aircraft based on the 11th information and the 12th information; And sending the 21st information from the server to the unmanned aerial vehicle.

Description

UAV landing induction method

The present invention relates to a method for guiding an unmanned aerial vehicle landing.

Currently, as the drone market, which is an unmanned flying vehicle, becomes active, related industries are expanding, such as delivery of goods using drones, pickup services, and provision of emergency supplies.

Although several related technologies are being developed, there are technical problems in accurately authenticating and recognizing service users who requested drone services due to GPS location errors or obstacles such as surrounding structures.

More specifically, when the receiver (user) location information is input, the drone flies toward the receiver through autonomous flight to the destination. However, with the current GPS technology, errors of 10~50m occur, especially when there are obstacles that obstruct the view of drones (for example, wooded forests) or when unspecified people including recipients are located at the destination (eg (Parks, etc.) has a difficult problem in identifying the recipient of the drone.

Also, when the drone, the server, and the recipient are connected and authenticated, the drone and the server recognize that the recipient exists nearby, but does not mean that the drone recognizes the recipient.

Therefore, after the recipient and the drone mutually verify (authentication), a detection and marking action must be made to recognize the recipient, and for this purpose, the drone can be moved into an open space where the drone is easily seen or without obstacles. It needs to be moved.

When examining patents related to user identification of drones in Korea, there is a'delivery method through user identification of unmanned delivery robot (No. 10-2017-0110341, publication date 2017.10.11)'. The present patent discloses the concept that if the unmanned delivery robot comes to a location near the location of the user of the destination, even if the location recognition is inaccurate, the user can be identified and a smooth and effective delivery service is possible without contacting the robot.

The prior user identification method relates to a technique in which a drone identifies a user remotely, and a method in which the drone specifies a user after mutual authentication between the drone and the user is not explicitly disclosed.

On the other hand, even after the drone recognizes the user, it is virtually impossible to land the drone exactly at the desired position. For example, even if the drone recognizes the user, the image processing and flight control technology has not developed to such an extent that the drone recognizes the flat land and safely lands away from the crowd including the user. 'Human interaction with unmanned aerial vehicles (U.S. Patent No.9459620, Registration Date 2016.10.4)' exists as a related prior patent document. Prior patent documents disclose that a person induces a drone through a gesture or the like. However, these gestures must be recognized through image processing and the like, and it is impossible to inform the drone of the correct landing point through the gestures. 19 discloses a method for inducing an unmanned aerial vehicle according to the related art.

Republic of Korea Patent Publication No. 10-2017-0110341 (2017.10.11) U.S. Patent No. 9459620 (2016.10.4)

It is an object of the present invention to provide a method for guiding an unmanned aerial vehicle landing. More specifically, it is to provide a method for a drone such as a drone to automatically land in response to a user when a user issues a command for a location such as a landing point.

According to an aspect of the present invention, the server receiving the eleventh information including the azimuth information of the wireless terminal from the wireless terminal; Receiving, by the server, twelfth information including angle information inclined with respect to the surface of the wireless terminal from the wireless terminal; Generating, by the server, 21st information including information about the landing point by calculating a landing point of the unmanned aircraft based on the 11th information and the 12th information; And sending the 21st information from the server to the unmanned aerial vehicle.

In addition, the server receiving the 31st information including the image of the landing point from the unmanned aerial vehicle; Transmitting, by the server, the 31st information to the wireless terminal; Receiving, by the server, thirteenth information including a landing point confirmation message from the wireless terminal; And transmitting, by the server, the 23rd information including the control signal of the drone to fly to the landing point.

In addition, the server may further include the step of transmitting a message to the wireless terminal requesting the wireless terminal to photograph the landing point.

Further, the twelfth information may be generated based on an imaginary straight line connecting the landing point of the wireless terminal and the unmanned aerial vehicle and angle information between a surface and a horizontal surface including the landing point.

In addition, according to another aspect of the present invention, the unmanned aircraft receiving a 51st from the wireless terminal including the azimuth information of the wireless terminal; Receiving, by the wireless terminal, 52th information including angle information inclined with respect to the ground surface by the wireless terminal from the wireless terminal; Calculating the landing point of the unmanned aerial vehicle based on the 51st information and the 52th information to generate 61st information including information on the landing location; And using the 61st information to fly to the landing point.

In addition, according to another aspect of the present invention, the unmanned aircraft receiving a 51st from the wireless terminal including the azimuth information of the wireless terminal; Receiving, by the wireless terminal, 52th information including angle information inclined with respect to the ground surface by the wireless terminal from the wireless terminal; Calculating the landing point of the unmanned aerial vehicle based on the 51st information and the 52th information to generate 61st information including information on the landing location; Generating the 71st information by photographing the landing point based on the 61st information; Transmitting the 71st information to the wireless terminal by the drone; The drone receiving 53rd information including a landing point confirmation message from the wireless terminal; And flying the drone to the landing point by utilizing the 53rd information and the 61th information.

In addition, the UAV may further include transmitting a message to the wireless terminal requesting the wireless terminal to photograph the landing point.

The 52th information may be generated based on an imaginary straight line connecting the landing point between the wireless terminal and the unmanned aerial vehicle and the angle information between the ground surface and the horizontal surface including the landing point.

The step of flying to the landing point may further include a step of generating a landing warning sound.

In addition, according to another aspect of the present invention, the wireless terminal transmitting the 81st information including the azimuth information of the wireless terminal to the server or the unmanned aerial vehicle; Transmitting the 82nd information including the angle information inclined with respect to the surface of the wireless terminal from the wireless terminal to the server or the unmanned wireless terminal by the wireless terminal; Receiving, by the wireless terminal, 91st information including an image of a landing point photographed from the server or the unmanned aerial vehicle; The wireless terminal may be a method for inducing landing of an unmanned aerial vehicle including the step of transmitting 101th information including a landing point confirmation message to the server or the unmanned aerial vehicle.

In addition, the wireless terminal further comprises the step of receiving a message from the server or the unmanned aerial vehicle requesting the wireless terminal to photograph the landing point of the unmanned aerial vehicle, wherein the azimuth and angle information include the azimuth and the angle information of the wireless terminal capturing the landing location. It may be a value at a point in time.

In addition, the 102th information may be generated based on angular information between a virtual straight line connecting the landing point of the wireless terminal and the unmanned aerial vehicle and an earth surface and a horizontal surface including the landing point.

According to an embodiment of the present invention can provide a method for inducing unmanned landing.

1 is a diagram illustrating a communication connection relationship between a server and a drone and a wireless terminal according to the present invention.
2 is a flow chart showing a method according to the invention in terms of a server.
3 is a conceptual diagram illustrating angle information inclined with respect to the ground surface of the wireless terminal.
4 is a view of a user holding a drone and a wireless terminal and a landing point viewed from a plane.
5 and 6 is a conceptual diagram showing the landing point in the wireless terminal.
7 is a flow chart showing a method according to the invention from the perspective of an unmanned aerial vehicle.
8 is a flow chart showing a method according to the invention from the perspective of a wireless terminal.
9 is a view showing a user's wireless terminal-server-unmanned operation and data transfer relationship.
10 is a flow chart showing a method according to the invention from the perspective of a server.
11 is a flowchart illustrating a method according to the present invention from the perspective of a wireless terminal.
12 to 13 are flow charts showing a method according to the invention from the perspective of an unmanned aerial vehicle.
14 to 16 are explanatory diagrams showing a mutual recognition method between an unmanned aerial vehicle and a wireless terminal according to an embodiment of the present invention.
17 is an exemplary view of a display screen for selecting an image portion for a specific target according to an embodiment of the present invention.
18 is a view showing an operation between a server, a UAV, and a user wireless terminal in an application to which the present invention is applied.
19 is a method for inducing an unmanned aerial vehicle according to the related art.

The present invention can be applied to various transformations and can have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all conversions, equivalents, and substitutes included in the spirit and scope of the present invention. In the description of the present invention, when it is determined that a detailed description of known technologies related to the present invention may obscure the subject matter of the present invention, the detailed description will be omitted.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from other components.

The terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, terms such as “include” or “have” are intended to indicate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, and that one or more other features are present. It should be understood that the existence or addition possibilities of fields or numbers, steps, operations, components, parts or combinations thereof are not excluded in advance.

Hereinafter, an embodiment according to the present invention will be described in detail with reference to the accompanying drawings, and in describing with reference to the accompanying drawings, identical or corresponding components are assigned the same reference numbers and redundant description thereof is omitted. Shall be

In addition, terms such as first and second used hereinafter are only identification symbols for distinguishing the same or corresponding components, and the same or corresponding components are limited by terms such as first and second. no.

In addition, the term "combination" means a contact relationship between each component, and does not mean only a case in which physical direct contact is made between each component, but other components are interposed between each component, so that the components are in different components. It should be used as a comprehensive concept until each contact is made.

Further, in connection with a computer program, the suffixes "modules" and/or "parts" for components are part of a computer program for functioning a computer as a means of a specific function, or a computer program for realizing a specific function in a computer. It can be a part. For example, the module A may be interpreted as a computer program for functioning the computer as means A or a computer program for realizing function A in the computer. As a method, the "step" may be implemented as a computer program in a computer and executed.

Meanwhile, an application refers to a set of computer programs designed to perform a specific task, and is also referred to as an application program. The user can add a related function by installing an application according to an embodiment of the present invention in his/her electronic device.

The electronic device of the user on which the application is installed consists of a CPU, RAM, ROM, storage device, such as a computer, tablet PC, or smartphone, and is an environment in which the entire system is controlled by a graphical operating system such as Windows, ios, Android, and Linux. It is preferable, and it is especially specialized in a smartphone that can send and receive calls and texts with registered contacts.

In addition, the flow chart of the drawings attached to the present specification is only a flow chart for explaining the invention, and need not be a flow chart for perfect implementation without bugs on a computer.

In addition, the terminal referred to in this specification may be a smartphone, a personal computer, a tablet PC, etc. as a general user terminal.

1 is a diagram illustrating a communication connection relationship between a server and a drone and a wireless terminal according to the present invention.

Referring to FIG. 1, the unmanned aerial vehicle 10, the user wireless terminal 20, and the server 30 can exchange data. In general, an unmanned aerial vehicle refers to an unmanned aerial vehicle in which a person does not board, but the implementation effect of the present invention can be applied to all flying mobile objects that require the user's exact location and recognition, so the unmanned aerial vehicle according to the present invention is a photographing means (eg, a camera Etc.), a geomagnetic sensor, a 3-axis acceleration sensor, a sensor such as GPS (GNSS) is built-in, so it can collectively refer to a vehicle having information such as location coordinates and azimuth.

The user according to the present invention can be any user receiving a service by the UAV, such as a recipient of delivery of the goods by the UAV, a pickup service user requesting the UAV service to deliver goods to another place, a user requesting a drone on demand shooting service, and the like. have.

In general, a server can exchange data with a UAV and a user wireless terminal through wireless communication, but is not necessarily limited to wireless communication, and data may be exchanged through wired communication. In this specification, description of the technology for exchanging data is omitted.

The server receives information received from the unmanned aerial vehicle and/or the user wireless terminal when the unmanned aerial vehicle approaches each other with a predetermined purpose for providing an unmanned aircraft utilization service such as delivery of goods to a user having the wireless terminal. Here, the information received from the user wireless terminal may be flight-related information, status information, user information, and location information of the unmanned aerial vehicle.

More specifically, the server transmits a command required for the operation of the UAV (eg, dispatch, authentication, guide, return, etc.) to the UAV based on the state information including the user's location information. In addition, the server continuously monitors the situation according to the presence or absence of the unmanned aerial vehicle, the identification of the flight path, the user's condition, etc., and lands on the basis of the relative location information of the user and the unmanned aerial vehicle, altitude information, azimuth information, etc. You can calculate the route adjustment data of the UAV required for the guide. In addition, the server may calculate the location of the landing point of the unmanned aircraft based on the user's location information and azimuth information when the unmanned aircraft lands and transmit the unmanned aerial vehicle to the unmanned aerial vehicle.

On the other hand, the UAV continuously transmits all status information and flight-related information of the aircraft including location, speed, altitude, azimuth, and battery status to the server. In addition, the UAV may transmit altitude, azimuth, and relative angle information to the server at the time of authentication with the user.

Meanwhile, the user wireless terminal transmits data and commands such as location information, angle, azimuth, and elevation information of the wireless terminal to the server. In addition, the user wireless terminal can display the unmanned information and the situation transmitted from the server.

2 is a flow chart showing a method according to the invention in terms of a server. Referring to Figure 2, the server receiving the eleventh information including the azimuth information of the wireless terminal from the wireless terminal (S110); Receiving, by the server, twelfth information including angle information inclined with respect to the surface of the wireless terminal from the wireless terminal (S120); The server generating 21st information including information on the landing point by calculating a landing point of the unmanned aircraft based on the 11th information and the 12th information (S130); The server transmitting the 21st information to the drone (S140); Receiving, by the server, 31st information including an image of the landing point photographed from the unmanned aerial vehicle (S150); The server transmitting the 31st information to the wireless terminal (S160); And receiving, by the server, thirteenth information including a landing point confirmation message from the wireless terminal (S170).

More specifically, referring to FIG. 3, the angle information inclined with respect to the surface of the wireless terminal is referred to as an angle formed by an imaginary straight line connecting the wireless terminal and the unmanned landing point and the surface including the landing point. Can be. When the wireless terminal held by the user is spaced vertically from the surface of the surface, that is, the height of the wireless terminal is generally within about 1.5m, and using the height and the angle information, the landing point is the wireless terminal, that is, You can calculate how far you are from the user. The angle information may be measured through various sensors such as a gyro sensor included in the wireless terminal.

4 is a plan view of a user having a drone and a wireless terminal and a landing point from a plane. Referring to FIG. 4, the UAV may recognize an absolute coordinate of the wireless terminal by recognizing a wireless terminal or a user passing through the wireless terminal. This will be described later. The basic premise that the UAV or the server recognizes or knows the absolute coordinates (eg GPS value, latitude/longitude value, etc.) of the wireless terminal is located, and the UAV or the server can receive the landing point information indicated by the UAV. As above, the UAV or the server can calculate the exact landing point using the azimuth angle between the wireless terminal and the landing point and the angle information generated by the wireless terminal (the angle between the surface and the wireless terminal-landing point). Although the term azimuth is used in this specification, it is not necessary to be limited to the term azimuth, and any information capable of relatively displaying the angle or position between the unmanned aerial vehicle, the wireless terminal, and the landing point can be used instead of the azimuth. In addition, the user photographs a desired landing point through a sensor such as a camera of the wireless terminal, and at this point in time, the wireless terminal may generate an azimuth angle of the wireless terminal. However, the concept of shooting in the present specification is not limited to, for example, an action of clicking a virtual shutter of a camera on a smartphone, and positioning the landing point in the viewfinder for a predetermined time (for example, 3 seconds). It is possible to include all concepts that can generate information such as azimuth through an imaginary line connecting a wireless terminal and a landing point, such as an action.

In addition, according to an embodiment of the present invention, a server may receive image information of the landing point from the unmanned aerial vehicle, and transmit it to the wireless terminal again. Alternatively, the wireless terminal may receive image information directly from the unmanned aerial vehicle. In addition, the server can be confirmed by the wireless terminal whether the image information is associated with the correct landing point. For example, the wireless terminal receives a photo including a landing point, and a variety of methods (eg, text, voice message) to determine whether a related application (eg, a landing-related application) contains a landing point desired by the user. You can query Through this, if the desired landing point is correct, it is possible to confirm that the landing point is correct by transmitting data to a server or a drone through a manipulation such as a click. In addition, referring to FIG. 5, as shown in the drawing, an image indicating the landing point calculated by the unmanned aerial vehicle or the server is transmitted to the wireless terminal, and the wireless terminal can confirm this. In addition, as shown in FIG. 6, when the user wants to change the landing point output to the wireless terminal, the user can change to the desired landing point through various methods such as clicking or dragging the landing point display (X). The server or the UAV receives the information about the changed landing point and can finally derive the modified landing point through recalculation. Calculation of the actual landing point through the analysis of the photographic image showing the changed landing point, measures the actual distance from the drone to the subject's landing point, the dragged length on the image from the first displayed landing point to the modified landing point, the first displayed on the photographic image It can be calculated from various data such as the number of pixels between the landing point and the modified landing point. As a result, through this, the unmanned aerial vehicle can confirm the calculated landing point again by the user, and then fly to the correct landing point or the modified landing point to safely land.

In addition, an embodiment according to the present invention may further include the step of transmitting the 23rd information including the control signal of the unmanned aerial vehicle to fly to the landing point after the step S170.

In addition, the server may further include the step of transmitting a message to the wireless terminal requesting the wireless terminal to photograph the landing point. This step may be performed simultaneously or before step S110, but is not limited thereto.

7 is a flow chart showing a method according to the invention from the perspective of an unmanned aerial vehicle. This will be described with reference to FIG. 7. However, parts that overlap with the previous description will be omitted. According to another embodiment of the present invention, the unmanned aircraft receiving a 51 st including azimuth information of the wireless terminal from the wireless terminal (S610); Receiving, by the wireless terminal, 52th information including angle information inclined with respect to the surface of the wireless terminal from the wireless terminal (S620); Calculating the landing point of the unmanned aircraft based on the 51st information and the 52th information to generate 61st information including information on the landing point (S630); Generating a 31st information by photographing the landing point based on the 61st information (S640); Transmitting the 31st information to the wireless terminal (S650); Receiving the 53rd information including a landing point confirmation message from the wireless terminal (S660); The step of flying the drone to the landing point using the 53rd information and the 61th information (S670) and the step of flying to the landing point may include generating a landing warning sound (S680).

Unlike the previous embodiment, this embodiment has a difference in performing a landing point request between a UAV and a wireless terminal without performing a server, and performing flight control to the landing point. Through this, inevitably, the drone can be safely landed by landing at the landing point through control between the UAV and the wireless terminal in an environment where the central server does not exist.

Of course, as in the previous embodiment, the UAV may receive information including the modified landing point from the wireless terminal, and re-compute the landing point to fly. In addition, while flying to the landing point, the landing warning sound may be generated, or the flight warning light with high visual recognition may be turned on to call attention to the user.

8 is a flow chart showing a method according to the invention from the perspective of a wireless terminal. This will be described with reference to FIG. 8. However, parts that overlap with the previous description will be omitted. According to another embodiment of the present invention, the wireless terminal transmits the eleventh information including the azimuth information of the wireless terminal to the server or the unmanned aerial vehicle (S810); Transmitting, by the wireless terminal, twelfth information including angle information inclined with respect to the surface of the wireless terminal from the wireless terminal to the server or the unmanned aerial vehicle (S820); The wireless terminal receiving the 71st information including the image of the landing point photographed from the server or the unmanned aerial vehicle (S830); The wireless terminal may include transmitting the 81st information including the landing point confirmation message to the server or the drone (S840). In addition, the other embodiment may further include the step (S800) of receiving a message requesting that the wireless terminal photographs the landing point of the unmanned aerial vehicle from the server or the unmanned aerial vehicle simultaneously or before S810. In addition, the azimuth angle and the angle information may be values before or after the point at which the wireless terminal photographs the landing point.

9 is a view showing a user's wireless terminal-server-unmanned operation and data transfer relationship. The description is omitted because the previous embodiment and the description are duplicated.

Hereinafter, in this specification, a description will be given of a method for a UAV to recognize a user or a terminal possessed by the user.

10 is a flow chart showing a method according to the invention from the perspective of a server. Referring to FIG. 10, when the image of the unmanned aerial vehicle is located in a predetermined section of the imaging surface of the image sensor in the wireless terminal, the server is provided with first state information of the wireless terminal based on information about the external magnetic field of the wireless terminal from the wireless terminal. Receiving (S1210); When the image of the unmanned aerial vehicle is located in a predetermined section of the imaging surface of the image sensor in the wireless terminal, a server receiving second state information of the wireless terminal based on information on the orientation of the wireless terminal from the wireless terminal (S1220). ; And the server transmits a signal to control the UAV to fly or tilt or rotate the camera so that the target of the target having the wireless terminal is located on the imaging surface of the camera in the UAV based on the first state information. It consists of step (S1230). The steps S1210 and S1220 need not be sequential, but may be changed or performed simultaneously. In addition, any one of the above steps S1210 and S1220 may be present.

Here, the wireless terminal may be a general smartphone that can be gripped by a general user by hand. Also, an image sensor converts an optical image formed by a lens into an electrical signal, and the image sensor is a charge coupled device (CCD), a metal oxide semi-conductor (MOS), or a complementary metal (CMOS). -oxide semiconductor, a complementary metal oxide semiconductor). However, the type of the image sensor is not limited thereto. In addition, the image may be a concept of a broad sense that includes not only a digital signal in which an optical signal is converted, but also a result of outputting this digital signal as light visualized through a display device.

In addition, the imaging surface refers to an area of a device that serves to receive light from a light collecting part of a camera, such as a CCD or CMOS sensor of a film or digital camera, and refers to a place where an image of a subject is formed in the image sensor. In addition, a predetermined section means a specific portion of the imaging surface, and may generally be about the center of the imaging surface. However, the predetermined section is not necessarily limited to the center. In addition, whether the image of the unmanned aerial vehicle is located in a predetermined section of the imaging surface may coincide with whether the unmanned aerial vehicle is located on the display screen in the user's wireless terminal, but the present invention is not limited thereto. In other words, even in the case of a terminal without a display screen (eg, a black box, etc.), if it is possible to determine whether the unmanned aerial vehicle is facing the terminal, it can be considered that it is included in the scope of the present invention. In the present invention, it is key to understand where the user terminal is located based on the unmanned aerial vehicle, and as an embodiment of the present invention, when the image of the unmanned aerial vehicle is imaged on the imaging surface of the wireless terminal, the unmanned aerial vehicle and the user terminal face each other. It simply states that it can be judged.

In addition, the relative position or direction of the UAV and the user terminal may be calculated when it is determined that the UAV and the user terminal face each other.

Next, the first state information may include external magnetic field information of the wireless terminal. Information about the external magnetic field can be the Earth's magnetic field. In more detail, it includes information on any property related to a magnetic field in general physics, including information about a magnetic field formed outside the wireless terminal (eg, size and direction). In addition, when the terminal of the present specification measures an external magnetic field, the magnetic field rapidly changes is a concept included in the external magnetic field information. For example, a sudden and weakening of the external magnetic field is a concept of a change in the size of the magnetic field, which is included in the external magnetic field information. In addition, the rapidly changing direction of the magnetic field is a concept included in external magnetic field information. In other words, the information about the external magnetic field includes a concept of a change in the external magnetic field and includes a change in the magnitude or direction of the magnetic field over time. In addition, the first state information includes azimuth information, and may generally provide information on a two-dimensional plane. In addition, when using the first state information, it can be effectively used when the camera mounted in the unmanned aerial vehicle is already tilted downward.

Next, the second status information may include orientation information of the wireless terminal. Orientation information refers to the position information of the terminal, and may mean at least one of azimuth, tilt, and rotation. The orientation information of the present invention may indicate directional information in a three-dimensional space, and any attribute values indicating the direction of an object may be used. The orientation information of the wireless terminal may be obtained through an optical gyro sensor, a MEMS gyro sensor, etc., but is not limited thereto. In addition, the second state information includes directional information, and may generally provide information regarding a 3D plane.

In more detail, when the image of the unmanned aerial vehicle is located in a predetermined section of the imaging surface of the image sensor in the wireless terminal, the server is configured to provide a first of the wireless terminal based on information about the external magnetic field of the wireless terminal from the wireless terminal. Status information can be received.

In other words, when a drone such as a drone is captured by a camera of a wireless terminal such as a smartphone, the server may control the smartphone to measure an external magnetic field, that is, azimuth information. Through this, it is possible to measure the orientation of the wireless terminal at the moment the wireless terminal gazes in front of the unmanned aerial vehicle (for example, north and southwest).

On the other hand, when the image of the unmanned aerial vehicle is located in a predetermined section of the imaging surface of the image sensor in the wireless terminal, the server may receive the second status information of the wireless terminal based on the information on the orientation of the wireless terminal.

In other words, the server controls the smartphone to measure the orientation of the wireless terminal body relative to the orientation, i.e., the reference position of the drone, when the drone, such as a drone, is captured by the camera of the wireless terminal, such as a smartphone. Can. Through this, the relative direction between the unmanned aerial vehicle and the user's wireless terminal at the moment when the wireless terminal gazes in front of the unmanned aerial vehicle can be measured.

Next, the server that has received the first or second status information may control the UAV or the camera in the UAV based on the first or second status information. In more detail, the unmanned aerial vehicle may be hovered in a predetermined direction. For example, when the first unmanned aerial vehicle reaches the delivery point, it can hover in a predetermined direction (for example, toward the north). Thereafter, when the first or second status information is received through the server to the UAV or directly from the wireless terminal, the UAV calculates from which direction the wireless terminal is staring at the UAV by using the first or second status information. can do. Thereafter, the camera may be rotated or controlled to rotate so that the camera in the unmanned aerial vehicle can stare at the wireless terminal based on the result of the calculation. Through this, the camera of the unmanned aerial vehicle can stare directly at the wireless terminal.

11 is a flowchart illustrating a method according to the present invention from the perspective of a wireless terminal. This will be described with reference to FIG. 11. However, parts that overlap with the previous description will be omitted. Referring to FIG. 11, when the image of the unmanned aerial vehicle is located in a predetermined section of the imaging surface of the image sensor in the wireless terminal, the wireless terminal is a server or an unmanned aerial vehicle based on information about an external magnetic field of the wireless terminal. Transmitting status information (S1310); When the image of the unmanned aerial vehicle is located in a predetermined section of the imaging surface of the image sensor in the wireless terminal, the wireless terminal transmits the second status information of the wireless terminal based on the information on the orientation of the wireless terminal to the server or unmanned aircraft ( S1320); Receiving, by the wireless terminal, an image of a target having the wireless terminal taken by the unmanned aerial vehicle from the unmanned aerial vehicle or the server (S1330); The wireless terminal outputting the image to an output unit in the terminal (S1340); The wireless terminal selecting and inputting an image portion for a specific target existing in the image (S1350); And the wireless terminal transmitting the video portion to the server or the unmanned aerial vehicle (S1360). The steps S1310 and S1320 need not be sequential, but may be changed or performed simultaneously. In addition, any one of the above steps S1310 and S1320 may be present.

In more detail, when the image of the unmanned aerial vehicle is located in a predetermined section of the imaging surface of the image sensor in the wireless terminal, the wireless terminal is a server or unmanned aircraft based on information about the external magnetic field of the wireless terminal. The first status information of the wireless terminal can be transmitted.

On the other hand, when the image of the unmanned aerial vehicle is located in a predetermined section of the imaging surface of the image sensor in the wireless terminal, the wireless terminal is a server or an unmanned aerial vehicle based on information about the orientation of the wireless terminal based on information about the orientation of the wireless terminal. Can be sent.

Next, the wireless terminal may receive an image of a target having the wireless terminal taken by the unmanned aerial vehicle from the unmanned aerial vehicle or the server, and output the image to an output unit in the terminal.

In other words, as the smartphone possessed by the user transmits information about the external magnetic field or information about orientation to the unmanned aerial vehicle or the server, the unmanned aerial vehicle can know the approximate location and direction of the user. Thereafter, when the unmanned aerial vehicle reaches the user's space, the unmanned aerial vehicle photographs a predetermined range including the user, and the user can receive his or her image photographed from the unmanned aerial vehicle through a smartphone.

The photographed image received from the unmanned aerial vehicle may be output through a display screen in the wireless terminal possessed by the user. However, other objects or obstacles may exist in the output image as well as the user. For example, in a crowded place such as a park, several unspecified people may be included in the video. Alternatively, if there are many trees, such as forests, or a large number of tall buildings, such as downtown areas, obstacles such as trees or buildings may be included in the image. In this case, the UAV may have difficulty in identifying and recognizing who the user is through the image. Therefore, it is necessary for a user to select a video portion for a specific target (user) using a wireless terminal and notify the unmanned aerial vehicle. To this end, the user can select an image portion for a specific target from images displayed on the display screen in the wireless terminal. The selection input may be made through user manipulation such as screen touch and marking. The wireless terminal may transmit an image for a specific target selected to the server or the unmanned aerial vehicle, and through this, the unmanned aerial vehicle may recognize the user specified from the user wireless terminal.

12 to 13 are flow charts showing a method according to the invention from the perspective of an unmanned aerial vehicle. This will be described with reference to FIG. 12. However, parts that overlap with the previous description will be omitted. Referring to FIG. 12, the UAV can directly perform control such as UAV flight, camera rotation, or tilting from the status information of the wireless terminal. Specifically, the step of receiving the status information of the wireless terminal from the server or the wireless terminal unmanned (S1410); Controlling the UAV to fly or tilt the camera so that the image of the target having the wireless terminal is located on the imaging surface of the camera in the UAV based on the status information (S1420) ); Photographing a target having the wireless terminal (S1430); Transmitting the photographed image to the server or the wireless terminal (S1440); And controlling the unmanned aerial vehicle to fly along a target identified based on the image (S1450).

Referring to FIG. 13, the UAV may receive control signals such as UAV flight, camera rotation, or tilting calculated by the server from state information of the wireless terminal, and perform control according to a command of the server. Specifically, the drone receives a signal to control the flight of the unmanned aerial vehicle so that the target of the target having the wireless terminal is positioned on the imaging surface of the camera in the unmanned aerial vehicle based on the status information of the wireless terminal from the server, or to rotate or tilt the camera. Step S1510; Controlling the unmanned aerial vehicle to fly control or rotate or tilt the camera based on the signal (S1520); Photographing a target having the wireless terminal (S1530); Transmitting the photographed image to the server or the wireless terminal (S1540); And controlling the unmanned aerial vehicle to fly along a target identified based on the image (S1550).

In other words, the unmanned aerial vehicle may directly receive status information of the wireless terminal or may receive it through a server.

In addition, the UAV directly controls the flight of the UAV so that the image of the user who has the wireless terminal is located on the imaging surface of the camera in the UAV based on the received status information of the wireless terminal, or receives a control signal from the server and receives the control signal from the server. You can control the drone's flight.

In addition, the UAV is controlled to rotate or tilt the camera of the UAV so that the image of the user who has the wireless terminal is located on the imaging surface of the camera in the UAV based on the received status information of the wireless terminal, or by receiving a control signal from the server. It can be controlled to rotate or tilt the drone's camera according to the command.

When the image of the user who has the wireless terminal is located on the imaging surface in the unmanned aircraft by controlling the flight of the unmanned aircraft or rotating or tilting the camera of the unmanned aerial vehicle, the unmanned aerial vehicle targets the user having the wireless terminal and includes the user. It is possible to take an image in a predetermined range. Next, the UAV may transmit the captured image to a server or a wireless terminal.

In addition, the UAV can fly along the identified target based on the captured image. Through this, the user can direct the unmanned aerial vehicle to a place where the unmanned aerial vehicle is easily landed in order to receive services such as delivery of goods from the unmanned aerial vehicle.

14 to 16 are explanatory diagrams showing a mutual recognition method between an unmanned aerial vehicle and a user wireless terminal according to an embodiment of the present invention. Referring to FIG. 14, the UAV may move up to the user's proximity based on the status information of the user's wireless terminal. At this time, the unmanned aerial vehicle has a certain directionality in the process of moving to the user's location. After the unmanned aerial vehicle arrives near the user, the unmanned aerial vehicle can hover while maintaining a predetermined direction (for example, a flight direction just before arrival). However, the UAV immediately after arrival near the user only recognizes the approximate user location range, and does not know the relative position and direction between the UAV and the user's wireless terminal. In order for the UAV and the user to accurately recognize each other, the UAV needs to recognize in which direction the wireless terminal is staring at the UAV.

The process of recognizing the relative direction between the UAV and the user's wireless terminal will be described later with reference to FIGS. 15 to 16. When the unmanned aerial vehicle arrives above the user's location, the user can photograph the unmanned aerial vehicle hovering in the air using the wireless terminal he is holding. Preferably, after the user moves to an open space free of obstacles, an unmanned aerial vehicle shooting operation may be performed.

More specifically, when the image of the unmanned aerial vehicle is located in a predetermined section of the wireless terminal screen or an image plane of an image sensor in the wireless terminal (eg, in the center), the user may take an unmanned aerial vehicle through an input such as clicking a photographing button. However, since a photographing error (for example, hand tremor) may occur in a process of performing an operation such as clicking a photographing button by a user, preferably, the unmanned image is displayed on a specific surface (eg, a center) on an imaging surface or a photographing screen. At the moment of conclusion, the shooting operation can be automatically performed in the application itself. Through this photographing operation, the first state information and the second state information of the user wireless terminal may be transmitted to a server or an unmanned aerial vehicle. Here, the status information of the user wireless terminal may include the height of the user's wireless terminal, the tilt angle, and the photographing orientation angle.

The server or the UAV can calculate the relative azimuth of the user's wireless terminal compared to the UAV by comparing the state information of the user's wireless terminal and the position and azimuth information of the UAV at the time of arrival based on the transmitted state information of the wireless terminal. .

More specifically, at the time when the image of the unmanned aerial vehicle is formed on the imaging surface (or display screen) of the user's wireless terminal, the heading point of the unmanned aerial vehicle is compared by comparing the hovering azimuth angle (heading point angle) of the unmanned aerial vehicle with the shooting angle and orientation information of the user wireless terminal. It is possible to calculate the counterfeit position of the user relative to each other. Through this, the server or the UAV can recognize which direction the user is in when the UAV is the central point. In other words, the location and direction of a user who wants to receive the UAV service is specified, so that the UAV and the user wireless terminal can mutually recognize each other.

Thereafter, the server or the UAV may control the UAV flight (eg, the hovering direction) while the UAV is hovering, or may control the rotation or tilting of the camera in the UAV. Through this control, the UAV or the camera in the UAV and the user wireless terminal can stare at each other in front, which means that the UAV can locate a predetermined range of images including the user in the camera imaging surface of the UAV. However, in order to provide an accurate UAV service, an operation for one-to-one matching between the UAV and the user needs to be additionally performed.

To this end, the UAV may photograph a predetermined range including the user at a point in time when the UAV and the UAV face each other. Thereafter, the UAV may transmit the photographed image to the user's wireless terminal. The transmitted image may be displayed on the display screen in the user's wireless terminal, and the user may select an image portion for a specific target from the images displayed on the display screen in the wireless terminal.

17 is an exemplary view of a display screen for selecting an image portion for a specific target according to an embodiment of the present invention. Referring to FIG. 17, the user wireless terminal may receive a captured image related to a predetermined range including a user photographed by the unmanned aerial vehicle. In addition, it can be displayed on the display screen on the wireless terminal. The user can select an image portion for a specific target from images displayed on the display screen in the wireless terminal, and the selection input may be made through user manipulation such as screen touch and marking. Thereafter, the wireless terminal may transmit an image for a specific target selected to the server or the UAV through which the UAV and the user can be matched one-to-one.

In addition, the UAV can fly along the identified target based on the captured image. Through this, the user can direct the unmanned aerial vehicle to a place where the unmanned aerial vehicle is easily landed in order to receive services such as delivery of goods from the unmanned aerial vehicle.

18 is a view showing an operation between a server, a user wireless terminal, and an unmanned aerial vehicle in an application to which the present invention is applied. Referring to FIG. 18, when a service request from a user is received and an unmanned aerial vehicle equipped with an article or service arrives near the user, the unmanned aerial vehicle may transmit a notification to the server that the user has arrived near the user. Next, the server may notify the user terminal of the arrival of the unmanned aerial vehicle and simultaneously instruct the application to be executed. The application execution command may be guided to the user based on various functions that can be performed by the user's wireless terminal, such as text notification, voice notification, or general telephone type. Next, when the user executes the application through a specific operation (eg, touch, etc.) for accepting the notification, the network connection between the user terminal, the server, and the unmanned aerial vehicle can be approved. When the network connection approval is completed, the server may request the user terminal to check the location of the UAV and photograph the UAV. Next, when the user terminal photographs the unmanned aerial vehicle and transmits the photographed image to the server, the server may calculate counterpart angle information between the unmanned aerial vehicle and the user's wireless terminal using the terminal status information included in the photographed image. In addition, the server may transmit the rotation/tilting control signal of the camera within the unmanned aircraft or the direction of the unmanned aerial vehicle being hovered in the vicinity of the user, according to the calculated counterpart angle information. The UAV may rotate to face the user terminal in front of the control signal of the server. Next, the server may request the UAV to check the location of the user terminal and photograph a predetermined range including the user. Next, when the UAV photographs a predetermined range including the user and transmits the captured image to the server, the server transmits it to the user terminal and simultaneously selects (targets) a specific target in the transmitted user captured image. You can ask. Next, the user can select a video portion for a specific target (for example, the user himself) in the user captured image received through manipulation such as screen touch and marking, and resend it to the server. Next, the server transmits the target-specified image to the unmanned aerial vehicle, and then commands the unmanned aerial vehicle to track the specified target.

The steps of the embodiment of the invention disclosed in FIG. 9 may be combined subsequent to the steps disclosed in FIG. 18.

The methods and processing described above are, for example, instructions for execution by a processor, controller, or other processing device, encoded or compact disk read only memory (CDROM), magnetic or optical disk, flash memory, random access memory (RAM) or read-only memory (ROM), erasable programmable read-only memory (EPROM) or other machine-readable media such as computer readable media.

Such media may be embodied as any device that contains, stores, communicates, propagates, or moves executable instructions for use by or in connection with an instruction executable system, apparatus, or device. Alternatively or additionally, as analog or digital logic using hardware, such as one or more integrated circuits, or one or more processor execution instructions; Or as an application programming interface (API) or Dynamic Link Library (DLL), software of functions defined as local or remote procedure calls or available in shared memory; Or it may be implemented as a combination of hardware and software.

In other implementations, the method may be represented as a signal or propagation-signal medium. For example, instructions that implement the logic of any given program may take the form of electrical, magnetic, optical, electromagnetic, infrared, or other types of signals. The system described above receives these signals at a communication interface, such as a fiber optic interface, antenna, or other analog or digital signal interface, restores instructions from the signal, stores them in machine readable memory, and/or uses a processor You can also run them.

Embodiments of the invention may include a carrier wave with electronically readable control signals, which can be operated with a programmable computer system on which one of the methods described herein is executed. Embodiments of the invention can be implemented as a computer program product having program code, the program code being operated to execute one of the methods when the computer program is running on a computer. The program code can be stored, for example, on a machine-readable carrier. One embodiment of the present invention may be a computer program having program code for executing one of the methods described herein when the computer program runs on a computer. The present invention may include a computer or a programmable logic device for executing one of the methods described above. Programmable logic devices (eg, field programmable gate arrays, complementary metal oxide semiconductor based logic circuits) may be used to perform some or all of the functions described above.

As described above, one embodiment of the present invention has been described, but those skilled in the art can add, change, delete, or add components within the scope of the present invention as described in the claims. It will be said that the present invention can be variously modified and changed by the like, and this is also included within the scope of the present invention.

10: UAV
20: user wireless terminal
30: server

Claims (12)

Receiving, by the server, eleventh information including azimuth information of the wireless terminal from the wireless terminal;
Receiving, by the server, twelfth information including angle information inclined with respect to the ground surface by the wireless terminal from the wireless terminal;
Calculating, by the server, a landing point of the unmanned aircraft based on the eleventh information and the twelfth information to generate twenty-first information including information on the landing point; And
And the server transmitting the 21st information to the drone,
The twelfth information is generated based on an angular information between an imaginary straight line connecting the wireless terminal and the landing point of the unmanned aerial vehicle and an earth surface and a horizontal surface including the landing point.
According to claim 1,
Receiving, by the server, 31st information including an image of the landing point photographed from the unmanned aerial vehicle;
Transmitting, by the server, the 31st information to the wireless terminal;
Receiving, by the server, thirteenth information including a landing point confirmation message from the wireless terminal; And
And sending, by the server, the 23rd information including a control signal of the unmanned aerial vehicle to fly to the landing point to the unmanned aerial vehicle.
According to claim 1,
And sending, by the server, a message requesting the wireless terminal to photograph the landing point to the wireless terminal.
delete delete delete delete delete delete Transmitting the 81st information including azimuth information of the wireless terminal to the server or the unmanned wireless terminal;
Transmitting, by the wireless terminal, 82th information including angle information inclined with respect to the ground surface by the wireless terminal from the wireless terminal to the server or the unmanned aerial vehicle;
The wireless terminal receiving the 91st information including the image of the landing point photographed from the server or the unmanned aerial vehicle;
The wireless terminal includes the step of transmitting 101 information including a landing point confirmation message to the server or the drone,
The 82nd information is a method for inducing unmanned landing, characterized in that it is generated based on angular information between a virtual straight line connecting the wireless terminal and the landing point of the unmanned aerial vehicle and an earth surface and a horizontal surface including the landing point.
The method of claim 10,
The wireless terminal further comprises receiving a message from the server or the unmanned aerial vehicle requesting the wireless terminal to photograph the landing point of the unmanned aerial vehicle,
The azimuth and the angle information is a method for inducing a landing of an unmanned aerial vehicle, characterized in that the wireless terminal is a value at a time when the landing point is photographed.
delete

Images