WO2018145650A1

WO2018145650A1 - Aircraft and control method therefor

Info

Publication number: WO2018145650A1
Application number: PCT/CN2018/075877
Authority: WO
Inventors: 胡华智; 孙海洋
Original assignee: 亿航智能设备（广州）有限公司
Priority date: 2017-02-08
Filing date: 2018-02-08
Publication date: 2018-08-16
Also published as: CN106774947A

Abstract

An aircraft and a control method therefor, the method comprising the following steps: acquiring an image by means of a binocular camera (10); carrying out depth processing on the image acquired by the binocular camera to acquire a depth image (20); controlling the safe distance between an aircraft and a person according to the depth image and parameters of the binocular camera (30); recognizing a gesture of the person in the depth image, and acquiring a control instruction of the aircraft according to the recognized gesture of the person (40); and controlling the aircraft within the safe distance according to the control instruction of the aircraft (50). With the aircraft and the control method therefor, it is possible to control the aircraft by means of human gestures, thereby facilitating use by a user, reducing the purchase cost of the user, increasing the amusement in controlling the aircraft and enhancing the interaction between the user and the aircraft.

Description

Aircraft and control method thereof

Technical field

The present invention relates to the field of aircraft technology, and in particular, to an aircraft and a control method thereof.

Background technique

Because UAV aircraft have the advantages of good maneuverability, low cost and convenient use, UAV aircraft have been used in many industries, such as aerial photography, agricultural plant protection, surveying and mapping, etc.

technical problem

In the process of implementing the present invention, the inventors have found that the prior art has the following problems: the existing drones are controlled by a remote control device or a mobile device such as a mobile phone. The advantage is that it can realize accurate and real-time control of the drone, and the reliability is high; the disadvantage is that the additional cost is added and it is not conducive to carrying, and the operation is complicated, the problem is difficult to get started, and there are certain requirements for the novice.

Technical solution

The main object of the present invention is to provide an aircraft and a control method thereof, which aim to solve the problems of the prior art.

To achieve the above objective, a first aspect of an embodiment of the present invention provides a method for controlling an aircraft, the method comprising the steps of:

Acquiring images through a binocular camera;

Performing depth processing on the image captured by the binocular camera to obtain a depth image;

Controlling a safe distance between the aircraft and a person according to the depth image and parameters of the binocular camera;

Identifying a gesture of a person in the depth image, and acquiring a control instruction of the aircraft according to the gesture of the recognized person;

The aircraft is controlled within the safe distance in accordance with a control command of the aircraft.

Further, the recognizing the gesture of the person in the depth image, and acquiring the control instruction of the aircraft according to the gesture of the recognized person includes:

Performing a segmentation process on the depth image to obtain a human hand image;

Using the template matching or the cascade classifier to perform gesture category recognition on the human hand image;

A control instruction of the aircraft is obtained according to the identified gesture category.

Further, the gesture category includes a static gesture and a dynamic gesture.

Further, the aircraft's control commands include six degrees of freedom control of the aircraft, aircraft start and stop landing control, and control of specific functions.

Further, the control of the specific function includes control of a camera camera function and control of a camera recording function.

Further, the recognizing the gesture of the person in the depth image, before acquiring the control instruction of the aircraft according to the gesture of the recognized person, further includes the steps of:

Performing fusion processing on the image acquired by the binocular camera to obtain a fused image;

Identifying and tracking a face in the fused image to determine a face position in the fused image;

Adjusting the aircraft or pan/tilt angle according to the position of the face in the fused image within the safe distance ensures that the face position is located in the fused image center area.

In addition, in order to achieve the above object, a second aspect of the embodiments of the present invention provides an aircraft, where the aircraft includes a binocular camera, a depth image processing module, a safety distance processing module, a gesture recognition module, and a control module;

The binocular camera is configured to acquire an image;

The depth image processing module is configured to perform depth processing on the image captured by the binocular camera to obtain a depth image;

The safety distance processing module is configured to control a safety distance between the aircraft and a person according to the depth image and parameters of the binocular camera;

The gesture recognition module is configured to identify a gesture of a person in the depth image, and acquire a control instruction of the aircraft according to the gesture of the recognized person;

The control module is configured to control the aircraft according to a control instruction of the aircraft within the safety distance.

Further, the gesture recognition module includes a segmentation processing module, a gesture category recognition module, and an acquisition module;

The segmentation processing module is configured to perform segmentation processing on the depth image to obtain a human hand image;

The gesture category identifying module is configured to perform gesture category recognition by using a template matching or a cascade classifier on the human hand image;

The acquiring module is configured to acquire a control instruction of the aircraft according to the identified gesture category.

Further, the gesture category includes a static gesture and a dynamic gesture.

Further, the aircraft further includes an image fusion processing module, a face recognition tracking module, and a face location processing module;

The image fusion processing module is configured to perform fusion processing on the image captured by the binocular camera to obtain a fused image;

The face recognition tracking module is configured to identify and track a face in the fused image, and determine a face position in the fused image;

The face position processing module is configured to adjust an aircraft or a pan/tilt angle according to a face position in the fused image within the safety distance to ensure that the face position is located in the fused image center area.

Beneficial effect

The aircraft and the control method thereof according to the embodiments of the present invention, after performing deep processing on the image captured by the binocular camera, identify the gesture of the person in the depth image, and obtain the control instruction of the aircraft according to the recognized gesture of the person, and at a safe distance The aircraft is controlled according to the control instructions of the aircraft. Therefore, the aircraft is controlled by human gestures, which is convenient for the user to use, reduces the user's purchase cost, increases the control fun of the aircraft, and enhances the interaction between the person and the aircraft.

DRAWINGS

1 is a schematic flow chart of an aircraft control method according to an embodiment of the present invention;

2 is a schematic diagram of a gesture recognition process of an aircraft control method according to an embodiment of the present invention;

3 is another schematic flowchart of an aircraft control method according to an embodiment of the present invention;

4 is a schematic structural view of an aircraft according to an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a gesture recognition module of an aircraft according to an embodiment of the present invention; FIG.

FIG. 6 is another schematic structural view of an aircraft according to an embodiment of the present invention.

The implementation, functional features, and advantages of the present invention will be further described in conjunction with the embodiments. .

Embodiments of the invention

It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

A mobile terminal embodying various embodiments of the present invention will now be described with reference to the accompanying drawings. In the following description, the use of suffixes such as "module", "component" or "unit" for indicating an element is merely an explanation for facilitating the present invention, and does not have a specific meaning per se. Therefore, "module" and "component" can be used in combination.

As shown in FIG. 1, a first embodiment of the present invention provides a method for controlling an aircraft, the method comprising the steps of:

10. Capture images through a binocular camera.

In the embodiment of the invention, the aircraft is provided with two cameras to form a set of binocular cameras. The two cameras can be set side by side or side by side. Of course, it can also be staggered, that is, the two cameras are not on the same horizontal line or vertical line. The two cameras are separated by a certain distance. In theory, the larger the separation distance, the better.

In this step, images are acquired simultaneously (synchronously) by the binocular camera, and the acquired images may be photos or video streams.

11. Perform deep processing on the image captured by the binocular camera to obtain a depth image.

Specifically, when the binocular cameras are arranged side by side, the two images collected by the two cameras are spliced side by side. In one embodiment, the image captured by the left camera can be spliced to the left, the image captured by the right camera is spliced to the right, and finally a stereo image of the left and right format is obtained; when the binocular camera is arranged side by side, the two cameras are The collected images are spliced together side by side. In one embodiment, the image captured by the upper camera can be spliced to the upper side, and the image captured by the lower camera is spliced to the lower side, and finally a stereoscopic image of the top and bottom format is obtained.

Further, before performing image splicing, the aircraft first performs resolution reduction processing on the original image, and then splicing the reduced resolution image to reduce the final stereoscopic image size, thereby avoiding excessive bandwidth consumption in subsequent transmission. Resources, thereby increasing the transmission speed and improving the real-time performance of image transmission.

Taking the captured image as a video stream as an example, the aircraft first samples the two video streams collected by the two cameras into two preset resolution video streams, and then splicing the two preset resolution video streams into one stereo. Video stream, where the preset resolution is lower than the original resolution. For example, the two cameras of the aircraft each capture a 4K resolution video stream, and the aircraft first samples two 4K resolution video streams into two 720P format video streams, and then streams the two 720P format video streams (or Up and down) stitching into a stereo video stream with a resolution of 2560*720 in the left and right format (or top and bottom format).

In addition, the aircraft can also save the original images captured by the two cameras in a local storage space. Further, before the saving, the original image is also compressed to save storage space, such as compressing the video stream into a H.265 format video file.

12. Control the safety distance between the aircraft and the person according to the depth image and the parameters of the binocular camera.

In this embodiment, in order to ensure a safe distance between the aircraft and the person, the safety distance between the aircraft and the person is controlled according to the depth image and the parameters of the binocular camera. Parameters can include focal length, accuracy, distortion factor, and more.

13. Identifying a gesture of a person in the depth image, and acquiring a control instruction of the aircraft according to the gesture of the recognized person.

Please refer to FIG. 2, in this embodiment, the step may include:

131. Perform a segmentation process on the depth image to obtain a human hand image.

132. Using a template matching or a cascade classifier to perform gesture category recognition on the human hand image.

133. Acquire a control instruction of the aircraft according to the identified gesture category.

In this embodiment, the gesture categories include static gestures and dynamic gestures.

In the present embodiment, the aircraft's control commands include six degrees of freedom control of the aircraft, aircraft start and stop landing control, and control of specific functions. The control of specific functions includes the control of the camera function and the control of the camera recording function.

It should be noted that the six degrees of freedom refer to the forward, backward, leftward, rightward, upward and downward flight of the aircraft.

As an example, when the recognized gesture is “index finger facing down”, the corresponding control aircraft may be landed; when the recognized gesture is “Victory” (ie, the index finger and the middle finger), the camera camera function of the aircraft may be correspondingly controlled; When the recognized gesture is "palm", the camera recording function of the aircraft can be controlled correspondingly; when the recognized gesture is "thumbs up", the corresponding camera of the controlling aircraft can stop the recording function; when the recognized gesture is "fist" "When the aircraft is controlled, it can fly in six degrees of freedom.

14. Control the aircraft within a safe distance according to the aircraft's control commands.

Please refer to FIG. 3, further, in this embodiment, before step 13, the method may further include the following steps:

121. Perform fusion processing on the image captured by the binocular camera to obtain the fused image.

122. Identify and track a face in the fused image to determine a face position in the fused image.

123. Adjust the position of the aircraft or the pan/tilt according to the position of the face in the fused image within the safe distance to ensure that the position of the face is located in the central area of the fused image.

The aircraft control method provided by the embodiment of the present invention, after performing deep processing on the image captured by the binocular camera, identifies a gesture of a person in the depth image, acquires a control instruction of the aircraft according to the recognized gesture of the person, and is within a safe distance according to the method. The aircraft's control commands control the aircraft. Therefore, the aircraft is controlled by human gestures, which is convenient for the user to use, reduces the user's purchase cost, increases the control fun of the aircraft, and enhances the interaction between the person and the aircraft.

Referring to FIG. 4, FIG. 4 is an aircraft according to a second embodiment of the present invention. The aircraft includes: a binocular camera 20, a depth image processing module 21, a safety distance processing module 22, a gesture recognition module 23, and a control module 24;

The binocular camera 20 is used to acquire images.

The images are acquired simultaneously (synchronously) by the binocular camera, and the acquired images may be photos or video streams.

The depth image processing module 21 is configured to perform depth processing on the image captured by the binocular camera 20 to obtain a depth image.

The safety distance processing module 22 is configured to control the safety distance between the aircraft and the person according to the depth image and the parameters of the binocular camera.

The gesture recognition module 23 is configured to identify a gesture of a person in the depth image, and acquire a control instruction of the aircraft according to the gesture of the recognized person.

Please refer to FIG. 5, in this embodiment, the gesture recognition module 23 may include a segmentation processing module 231, a gesture category identification module 232, and an acquisition module 233;

The segmentation processing module 231 is configured to perform segmentation processing on the depth image to obtain a human hand image.

The gesture category identification module 232 is configured to perform gesture category recognition on the human hand image by using a template matching or a cascade classifier.

The obtaining module 233 is configured to acquire a control instruction of the aircraft according to the identified gesture category.

The control module 24 is configured to control the aircraft according to a control instruction of the aircraft within a safe distance.

Please refer to FIG. 6, further, in this embodiment, the aircraft further includes an image fusion processing module 221, a face recognition tracking module 222, and a face location processing module 223;

The image fusion processing module 221 is configured to perform fusion processing on the image captured by the binocular camera to obtain the fused image;

The face recognition tracking module 222 is configured to identify and track a face in the fused image, and determine a face position in the fused image;

The face position processing module 223 is configured to adjust an aircraft or a pan/tilt angle according to a face position in the fused image within a safe distance to ensure that the face position is located in the fused image center area.

The aircraft provided by the embodiment of the present invention, after performing deep processing on the image captured by the binocular camera, identifies a gesture of a person in the depth image, acquires a control instruction of the aircraft according to the recognized gesture of the person, and is within a safe distance according to the aircraft Control commands to control the aircraft. Therefore, the aircraft is controlled by human gestures, which is convenient for the user to use, reduces the user's purchase cost, increases the control fun of the aircraft, and enhances the interaction between the person and the aircraft.

It is to be understood that the term "comprises", "comprising", or any other variants thereof, is intended to encompass a non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes those elements. It also includes other elements that are not explicitly listed, or elements that are inherent to such a process, method, article, or device. An element that is defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device that comprises the element.

The serial numbers of the embodiments of the present invention are merely for the description, and do not represent the advantages and disadvantages of the embodiments.

The above are only the preferred embodiments of the present invention, and are not intended to limit the scope of the invention, and the equivalent structure or equivalent process transformations made by the description of the present invention and the drawings are directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of the present invention.

Industrial applicability

The aircraft and the control method thereof according to the embodiments of the present invention, after performing deep processing on the image captured by the binocular camera, identify the gesture of the person in the depth image, and obtain the control instruction of the aircraft according to the recognized gesture of the person, and at a safe distance The aircraft is controlled according to the control instructions of the aircraft. Therefore, the aircraft is controlled by human gestures, which is convenient for the user to use, reduces the user's purchase cost, increases the control fun of the aircraft, and enhances the interaction between the person and the aircraft. Therefore, it has industrial applicability.

Claims

A method of controlling an aircraft, the method comprising the steps of:

Acquiring images through a binocular camera;

Performing depth processing on the image captured by the binocular camera to obtain a depth image;

Controlling a safe distance between the aircraft and a person according to the depth image and parameters of the binocular camera;

Identifying a gesture of a person in the depth image, and acquiring a control instruction of the aircraft according to the gesture of the recognized person;

The aircraft is controlled within the safe distance in accordance with a control command of the aircraft.
The control method of an aircraft according to claim 1, wherein the recognizing a gesture of a person in the depth image, and acquiring a control instruction of the aircraft according to the gesture of the recognized person comprises:

Performing a segmentation process on the depth image to obtain a human hand image;

Using the template matching or the cascade classifier to perform gesture category recognition on the human hand image;

A control instruction of the aircraft is obtained according to the identified gesture category.

3. The control method of an aircraft according to claim 2, wherein the gesture category comprises a static gesture and a dynamic gesture.

4. The control method of an aircraft according to any one of claims 1 to 3, wherein the control command of the aircraft comprises six degrees of freedom control of the aircraft, aircraft start/stop landing control, and control of a specific function.

5. The control method of an aircraft according to claim 4, wherein the control of the specific function comprises control of a camera camera function and control of a camera recording function.

The method for controlling an aircraft according to claim 5, wherein the step of identifying a gesture of a person in the depth image and obtaining a control instruction of the aircraft according to the gesture of the recognized person further comprises the steps of:

Performing fusion processing on the image acquired by the binocular camera to obtain a fused image;

Identifying and tracking a face in the fused image to determine a face position in the fused image;

Adjusting the aircraft or pan/tilt angle according to the position of the face in the fused image within the safe distance ensures that the face position is located in the fused image center area.

7. An aircraft comprising a binocular camera, a depth image processing module, a safety distance processing module, a gesture recognition module, and a control module;

The binocular camera is configured to acquire an image;

The depth image processing module is configured to perform depth processing on the image captured by the binocular camera to obtain a depth image;

The safety distance processing module is configured to control a safety distance between the aircraft and a person according to the depth image and parameters of the binocular camera;

The gesture recognition module is configured to identify a gesture of a person in the depth image, and acquire a control instruction of the aircraft according to the gesture of the recognized person;

The control module is configured to control the aircraft according to a control instruction of the aircraft within the safety distance.

8. The aircraft of claim 7, wherein the gesture recognition module comprises a segmentation processing module, a gesture category identification module, and an acquisition module;

The segmentation processing module is configured to perform segmentation processing on the depth image to obtain a human hand image;

The gesture category identifying module is configured to perform gesture category recognition by using a template matching or a cascade classifier on the human hand image;

The acquiring module is configured to acquire a control instruction of the aircraft according to the identified gesture category.

9. An aircraft according to claim 8, wherein the gesture categories comprise static gestures and dynamic gestures.

10. An aircraft according to any of claims 7-9, wherein the aircraft's control commands include six degrees of freedom control of the aircraft, aircraft start/stop landing control, and control of specific functions.

11. An aircraft according to claim 10, wherein the control of the particular function comprises control of a camera camera function, control of a camera recording function.

12. The aircraft of claim 11, wherein the aircraft further comprises an image fusion processing module, a face recognition tracking module, and a face location processing module;

The image fusion processing module is configured to perform fusion processing on the image captured by the binocular camera to obtain a fused image;

The face recognition tracking module is configured to identify and track a face in the fused image, and determine a face position in the fused image;

The face position processing module is configured to adjust an aircraft or a pan/tilt angle according to a face position in the fused image within the safety distance to ensure that the face position is located in the fused image center area.