WO2021043333A1

WO2021043333A1 - Photography control method and apparatus, aircraft, and flight system

Info

Publication number: WO2021043333A1
Application number: PCT/CN2020/123057
Authority: WO
Inventors: 冯银华
Original assignee: 深圳市道通智能航空技术有限公司
Priority date: 2019-09-05
Filing date: 2020-10-23
Publication date: 2021-03-11
Also published as: CN110650287A

Abstract

Disclosed in the present invention are a photography control method and apparatus, an aircraft, and a flight system, the photography control method being used in the aircraft, the aircraft being provided with a photography apparatus, and the method comprising: recognising first limb features of a user; on the basis of the first limb features, generating a corresponding prompt command; sending the prompt command to a terminal device having a communication connection to the aircraft, such that the terminal device generates corresponding prompt information on the basis of the prompt command; acquiring a response command in response to the prompt information; and, on the basis of the response command, controlling the photography apparatus to execute a corresponding photography operation. Provided in the present invention is a better aircraft photography control method.

Description

Shooting control method, device, aircraft and flying system

Technical field

The present invention relates to the technical field of aircraft control, in particular to a shooting control method, device, aircraft and flight system.

Background technique

With the development of technology, the development of unmanned aerial vehicles is getting faster and faster, especially the use of unmanned aerial vehicles for aerial photography or videography, which allows users to obtain image data with unique perspectives.

Among them, the unmanned aerial vehicle is usually controlled by a flight controller or a mobile terminal device with control capability during the shooting process, and the control method is too single to meet the user's needs for controlling the camera in a variety of camera environments.

Therefore, how to better control the shooting of the aircraft has become a hot issue of research.

Summary of the invention

The main purpose of the present invention is to provide a shooting control method, device, aircraft and flight system, aiming to better realize the shooting control of the aircraft.

To achieve the above objective, the present invention provides a shooting control method applied to an aircraft, the aircraft is provided with a camera device, and the method includes:

Identify the user’s first physical feature;

Generating a corresponding prompt instruction according to the first limb feature;

Sending the prompt instruction to a terminal device communicatively connected with the aircraft, so that the terminal device generates corresponding prompt information according to the prompt instruction;

Acquiring a response instruction in response to the prompt information;

According to the response instruction, the camera device is controlled to perform a corresponding camera operation.

Preferably, the identifying the first physical feature of the user includes:

Acquiring an environment image captured by the camera device;

Determining the characteristic part of the user according to the environmental image;

Determining a characteristic image area in the environment image according to the characteristic part;

The first body feature of the user is recognized according to the feature image area.

Preferably, the characteristic part includes at least one of the following: the user's arms, shoulders, head, and hands.

Preferably, the prompt instruction includes a first prompt instruction and a second prompt instruction, and then the generating the corresponding prompt instruction according to the first body feature includes:

Judging whether the first limb feature is a preset limb feature;

When the first limb feature is the preset limb feature, the first prompt instruction is generated;

When the first limb feature is not the preset limb feature, the second prompt instruction is generated.

Preferably, the prompt information includes at least one of the following: voice prompt information, image prompt information, and text prompt information.

Preferably, the obtaining the response instruction in response to the prompt information includes:

Acquiring the response instruction issued by the user controlling the terminal device according to the prompt information.

Acquiring the second body feature of the user in response to the prompt information;

The corresponding response instruction is generated according to the second limb feature.

The present invention also provides a shooting control device. The shooting control device includes a camera module and a vision chip communicatively connected with the camera module, wherein the vision chip includes:

The feature recognition module is used to recognize the user's first body feature;

An instruction generation module, configured to generate a corresponding prompt instruction according to the first limb feature;

An instruction sending module, configured to send the prompt instruction to a terminal device communicatively connected with the shooting control apparatus, so that the terminal device generates corresponding prompt information according to the prompt instruction;

An instruction acquisition module for acquiring a response instruction in response to the prompt information; and

The control module is configured to control the camera module to perform a corresponding camera operation according to the response instruction.

The present invention also provides an aircraft that includes a fuselage, an arm connected to the fuselage, a power device provided on the arm, a camera device connected to the fuselage, and an aircraft provided on the aircraft. A visual chip of the body, the visual chip includes:

Memory and processor;

The memory is used to store a computer-executable shooting control program;

The processor is used to call a shooting control program executable by the computer to implement the aforementioned shooting control method.

The present invention also provides a flight system. The flight system includes an aircraft and a terminal device communicatively connected to the aircraft. The aircraft includes a fuselage, an arm connected to the fuselage, and a A power device, a camera device connected to the fuselage, and a vision chip provided on the fuselage, the vision chip including:

Memory and processor;

The memory is used to store a computer-executable shooting control program;

Compared with the prior art, the shooting control method provided by the present invention is applied to an aircraft, the aircraft is provided with a camera device, and the method includes: recognizing a user's first limb feature; and generating a corresponding prompt based on the first limb feature Instruction; sending the prompt instruction to a terminal device communicatively connected with the aircraft, so that the terminal device generates corresponding prompt information according to the prompt instruction; obtaining a response instruction in response to the prompt information; according to the response instruction Control the camera device to perform corresponding camera operations.

By using the user’s first body feature to generate a corresponding prompt instruction, and send the prompt instruction to the terminal device to generate corresponding prompt information on the terminal device, so that the user can send corresponding response instructions to the aircraft according to the prompt information to achieve The better shooting control of the camera device provided in the aircraft makes the shooting control of the aircraft more adaptable and can adapt to a variety of shooting environments.

Description of the drawings

Fig. 1 is a schematic structural diagram of a flight control system provided by an embodiment of the present invention;

2 is a flowchart of steps of a shooting control method provided by an embodiment of the present invention;

Fig. 3 is a detailed step flow chart of step S10 in Fig. 2;

Figure 4A is a schematic structural diagram of the aircraft performing corresponding operations according to the user's physical characteristics;

FIG. 4B is a flow chart of the aircraft extracting the user's first limb feature according to the environment image;

Figure 5 is a flow chart of the aircraft extracting the user's second limb feature according to the environment image;

6 is a schematic diagram of a block diagram structure of an aircraft provided by an embodiment of the present invention;

FIG. 7 is a schematic block diagram of a module structure of a photographing control device according to an embodiment of the present invention.

detailed description

In order to make the objectives, technical solutions, and advantages of the present invention clearer, the following further describes the present invention in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not used to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects, without having to use To describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in a sequence other than the content illustrated or described herein. In addition, the terms "including" and "having" and any variations of them are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those clearly listed. Those steps or units may include other steps or units that are not clearly listed or are inherent to these processes, methods, products, or equipment.

It should be noted that the descriptions related to "first", "second", etc. in the present invention are only for descriptive purposes, and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. . Therefore, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on what can be achieved by a person of ordinary skill in the art. When the combination of technical solutions is contradictory or cannot be achieved, it should be considered that such a combination of technical solutions does not exist. , Is not within the protection scope of the present invention.

The present invention provides a shooting control method, a device, an aircraft, and a flight system. The shooting control method is applied to an aircraft, and the aircraft is provided with a camera device. The method includes: identifying a user's first body feature; The first body feature generates a corresponding prompt instruction; sends the prompt instruction to a terminal device communicatively connected with the aircraft, so that the terminal device generates corresponding prompt information according to the prompt instruction; obtains information that responds to the prompt information Response instruction; according to the response instruction, the camera device is controlled to perform a corresponding camera operation.

Please refer to FIG. 1. FIG. 1 is a flight system 100 provided by the present invention. The flight system 100 includes an aircraft 10 and a terminal device 20 communicatively connected with the aircraft 10. The terminal device 20 is used to control the aircraft 10.

The aircraft 10 may be a rotary-wing aircraft, such as a quad-rotor aircraft, a hexa-rotor aircraft, or a fixed-wing aircraft.

The aircraft 10 includes a fuselage 101, an arm 102 connected to the fuselage 101, a power device 103 provided on the arm 102, and a control system (not shown) provided on the fuselage 101. The power unit 103 is used to provide thrust, lift, etc. for the flight of the aircraft 10. The control system is the central nervous system of the aircraft 10 and may include multiple functional units, such as a flight control system, an image recognition system, a path planning system, and other specific functions. system. Among them, the flight control system includes various sensors, such as gyroscopes, accelerometers, etc., and the flight control system is used to control the flight attitude of the aircraft 10. The path planning system is used to plan the flight path of the aircraft 10 based on the location of the tracking target, and instruct the flight control system to control the flight attitude of the aircraft 10 to make the aircraft 10 fly according to the designated path.

The image recognition system includes a camera device 104 connected to the body 101 and a vision chip arranged on the body 101. The camera device 104 is electrically connected to the vision chip. The camera device 104 is used to capture media data such as images or videos of the user. The vision chip It is used to identify the user's gestures from the media data, thereby generating corresponding control instructions to control the aircraft 10 or the corresponding equipment provided on the aircraft to perform corresponding operations. For example, a camera control instruction is generated to control the camera device 104 to perform a corresponding camera operation. The camera device 104 can be a high-definition digital camera or other camera device. The camera device 104 can be set at any suitable location that is convenient for shooting. In some embodiments, the camera device 104 is installed on the bottom of the body 101 via a pan-tilt. In some embodiments, the vision chip may also be provided on the arm 102.

The terminal device 20 may be a smart phone, a tablet computer, a computer, a remote control, and the like. The user may interact with the terminal device 20 through any suitable type of one or more user interaction devices, and these user interaction devices may be a mouse, a button, a touch screen, and the like. Between the aircraft 10 and the terminal device 20, a communication connection can be established through wireless communication modules separately provided in each, such as a signal receiver, a signal transmitter, etc., and data/commands can be uploaded or issued.

Please refer to FIG. 2. FIG. 2 is a shooting control method provided by the present invention. The method is applied to an aircraft 10 and executed by a vision chip of the aircraft 10. The method includes:

Step S10: Identify the user's first body feature.

Different body languages of the user preset in the aircraft 10 can control the aircraft 10 to perform different actions. Different body languages correspond to different body features. By recognizing the user's body features, the user needs to express the body language through the body features.

Referring to FIG. 3, in some embodiments, step S10 includes:

Step S101: Acquire an environment image captured by the camera device.

Exemplarily, the environmental image is an image acquired by a camera 104 provided on the aircraft 10. Wherein, the way for the camera device 104 to obtain the environmental image may be that after the aircraft 10 is started, the camera device 104 periodically takes pictures or takes real-time taking pictures according to a preset program. It may also be that the user sends a shooting instruction to the aircraft 10 through the terminal device 20, so that the camera device 104 can take a shot.

Step S102: Determine the characteristic part of the user according to the environmental image.

The characteristic part of the user may be at least any one of the user's body, legs, arms, shoulders, head, or hands.

The image recognition technology is used to determine the user image area in the environment image, and identify the user's characteristic part from the user image area.

Step S103: Determine a characteristic image area in the environment image according to the characteristic part.

The feature image area corresponding to the feature location is acquired according to the acquired feature location and the preset association relationship, and the preset association relationship is the correspondence relationship between the feature location and the feature image area.

For example, when the characteristic part is the user's arm, the corresponding characteristic image area is the environment image. The preset range where the arms are located is the characteristic image area corresponding to the arm.

Step S104: Identify the first body feature of the user according to the feature image area.

According to the characteristic image area, the corresponding characteristic image and the first limb characteristic corresponding to the characteristic image are obtained from the characteristic image database set in the memory of the aircraft 10.

Refer to FIGS. 4A-4B. Illustratively, taking the characteristic part of arm C as an example for description, the aircraft 10 recognizes the user image area A in the environment image obtained by the camera 104, and recognizes the characteristic part of the user according to the user image area A C. Determine the characteristic image area B in the environment image A according to the characteristic part C of the user.

The feature image area B is compared with the preset feature image database to obtain the feature image D from the preset feature image database, and the corresponding first limb feature is obtained according to the feature image D and the association relationship between the preset feature image and the first limb feature. A physical feature.

As shown in FIG. 4B, the user's first limb feature in the environment image obtained by the aircraft 10 is that the arms C are raised and both are at an angle θ with the horizontal plane.

Step S11: Generate a corresponding prompt instruction according to the first limb feature.

The aircraft 10 generates a corresponding prompt instruction according to the acquired first limb feature and a preset relationship, where the preset relationship is the corresponding relationship between the first limb feature and the prompt information, and the corresponding relationship can be obtained according to the first limb feature. Prompt instructions.

In some embodiments, the prompt instruction includes a first prompt instruction and a second prompt instruction, and the generation of a corresponding prompt instruction according to the first body feature includes:

Judging whether the first limb feature is a preset limb feature;

Exemplarily, the preset limb feature can be raised with both arms C and at an angle θ with the horizontal plane, 90°≥θ≥60°, and the preset limb feature can also be raised with a single arm C and present with the horizontal plane. The included angle θ, 90°≥θ≥60°, the preset limb feature can also be that the single arm C is raised and the included angle θ with the horizontal plane, where θ is less than 60°.

Setting preset body features in the aircraft 10 can control the camera device 104 to perform a corresponding camera operation, where the camera operation can be any one of taking a picture, taking a picture, stopping a picture, or stopping a picture.

When it is recognized from the environment image that the user’s first limb feature is that both arms C are raised and are at an angle θ with the horizontal plane, 90°≥θ≥60°, and the single arm C is raised at an angle θ with the horizontal plane , 90°≥θ≥60° or the preset limb feature can also be a single arm C raised and an angle θ with the horizontal plane, where θ is less than any one of 60°, the first prompt instruction is generated ；

When it is recognized from the environment image that the user's first limb feature is not a preset limb feature, the second prompt instruction is generated.

Step S12: Send the prompt instruction to a terminal device communicatively connected with the aircraft, so that the terminal device generates corresponding prompt information according to the prompt instruction.

The aircraft 10 sends the prompt instruction to the terminal device 20 communicatively connected with the aircraft 10, so that the terminal device 20 generates prompt information corresponding to the prompt instruction according to the prompt instruction. The prompt information may be voice prompt information, text prompt information, or image. At least any one of the prompt messages.

Step S13: Obtain a response instruction in response to the prompt information.

In some embodiments, the obtaining a response instruction in response to the prompt information includes:

Acquire a response instruction issued by the user to manipulate the terminal device according to the prompt information.

Exemplarily, if the user wants to control the camera device 104 to perform a photographing operation through his current body language, if the aircraft 10 recognizes that the user’s current body language is to control the camera device 104 to perform the photographing operation and send a corresponding prompt instruction to the terminal device 20, The terminal device 20 is caused to generate prompt information corresponding to the prompt instruction, for example, "whether to take a picture" is displayed on the terminal device 20. If the user confirms that his current body language will control the aircraft 10 to perform the correct operation, he controls the terminal device 20 so that the terminal device 20 sends a first response instruction to the aircraft 10. The user can control the terminal device 20 by Voice control can also be triggered by triggering a button set on the terminal device 20. The button can be a physical button or a virtual button generated by installing an APP, which is not limited here.

For example, if the user confirms that it is necessary to control the camera 104 to perform a photographing operation, the terminal device 20 can be manipulated so that the terminal device 20 sends a first response instruction to confirm the operation in accordance with the prompt information to the aircraft 10, so that the aircraft 10 controls the camera 104 to perform the operation. Camera operation.

If the user confirms that his current body language will control the operation performed by the aircraft 10 is wrong, he controls the terminal device 20 so that the terminal device 20 sends a second response instruction to the aircraft 10.

For example, if the user cancels the need to control the camera 104 to perform a photographing operation, the terminal device 20 can also be manipulated so that the terminal device 20 sends the aircraft 10 a second response instruction to cancel the operation according to the prompt information.

Exemplarily, if the user confirms that it is necessary to control the camera 104 to perform a photographing operation, the current posture is maintained, so that the aircraft 10 obtains the first response instruction to confirm the operation according to the prompt information.

If the user needs to cancel the current control of the camera 104 to perform the photographing operation, the current posture is changed to change the current body language, so that the aircraft 10 obtains the user's second body feature after the corresponding posture change, and cancels according to the second body feature acquisition. The second response instruction of the prompt information operation, where the second body language is different from the first body language, and the aircraft 10 can be controlled to generate a corresponding second response instruction to control the camera 104 to perform an operation corresponding to the second response instruction.

As shown in Figure 5, it is preset that the second limb feature is that arm C is raised with one hand and is at an angle θ with the horizontal plane, 90°≥θ≥60°, and the second limb feature corresponds to enable the aircraft 10 to generate a camera The device 104 cancels the second response instruction of the camera operation. The user can control the generation of a second response instruction that causes the camera 104 to cancel the camera operation by changing the current body language to the corresponding body language with the second body feature.

Step S14: Control the camera device to perform a corresponding camera operation according to the response instruction.

The aircraft 10 controls the camera device to perform a camera operation corresponding to the response instruction according to the acquired response instruction, where the camera operation may be any one of taking a picture, taking a picture, stopping taking a picture, or stopping a picture taking.

Please refer to FIG. 6. In some embodiments, the vision chip further includes a memory 105 and a processor 106.

The memory 105 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, and the like. The memory 105 may be an internal storage unit of the aircraft 10 in some embodiments, such as a hard disk of the aircraft 10. In other embodiments, the memory 105 may also be an external storage device of the aircraft 10, such as a plug-in hard disk, a smart media card (SMC), and a secure digital (SD) card equipped on the aircraft 10. Flash Card, etc. The memory 105 can be used not only to store application software and various data installed in the aircraft 10, such as computer-readable code of a shooting control program, etc., but also to temporarily store data that has been output or will be output.

In some embodiments, the processor 106 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip, and the processor 106 may call program codes stored in the memory 105 or The data is processed to execute the aforementioned shooting control method.

Referring to FIG. 7, the present invention also provides a photographing control device 30, which is communicatively connected with terminal equipment. The photographing control device 30 includes a camera module 304 and a vision chip 300 communicatively connected with the camera module 304, wherein the vision The chip 300 includes:

The feature recognition module 301 is used to recognize the user's first body feature;

The instruction generation module 302 is configured to generate a corresponding prompt instruction according to the first limb feature;

The instruction sending module 303 is configured to send the prompt instruction to a terminal device communicatively connected with the shooting control apparatus, so that the terminal device generates corresponding prompt information according to the prompt instruction;

The instruction acquisition module 305 is configured to acquire a response instruction in response to the prompt information; and

The control module 306 is configured to control the camera module to perform a corresponding camera operation according to the response instruction.

In some embodiments, the feature recognition module 301 is also used to:

Acquiring an environment image captured by the camera device;

The terminal device can be a remote control device, a smart phone, a tablet computer, or a notebook computer.

In some embodiments, the characteristic part includes at least one of the following: voice prompt information, image prompt information, and text prompt information.

In some embodiments, the prompt instruction includes a first prompt instruction and a second prompt instruction, and the instruction generation module 302 is further configured to:

Judging whether the first limb feature is a preset limb feature;

If the first limb feature is the preset limb feature, correspondingly generate the first prompt instruction;

If the first limb feature is not the preset limb feature, the second prompt instruction is generated correspondingly.

In some embodiments, the prompt information includes at least one of the following: voice prompt information, image prompt information, and text prompt information.

In some embodiments, the instruction acquisition module 305 is also used to:

The above are only preferred embodiments of the present invention, and do not limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the description and drawings of the present invention, or directly or indirectly applied to other related technical fields , The same reason is included in the scope of patent protection of the present invention.

Claims

A photographing control method applied to an aircraft, characterized in that the aircraft is provided with a camera device, and the method includes:

Identify the user’s first physical feature;

Generating a corresponding prompt instruction according to the first limb feature;

Sending the prompt instruction to a terminal device communicatively connected with the aircraft, so that the terminal device generates corresponding prompt information according to the prompt instruction;

Acquiring a response instruction in response to the prompt information;

According to the response instruction, the camera device is controlled to perform a corresponding camera operation.
8. The method of claim 1, wherein said identifying said first physical feature of a user comprises:

Acquiring an environment image captured by the camera device;

Determining the characteristic part of the user according to the environmental image;

Determining a characteristic image area in the environment image according to the characteristic part;

The first body feature of the user is recognized according to the feature image area.
The method according to claim 2, wherein the characteristic part includes at least one of the following:

The user's arms, shoulders, head, and hands.
3. The method of claim 2, wherein the prompt instruction includes a first prompt instruction and a second prompt instruction, and then the generating the corresponding prompt instruction according to the first physical feature includes:

Judging whether the first limb feature is a preset limb feature;

When the first limb feature is the preset limb feature, the first prompt instruction is generated;

When the first limb feature is not the preset limb feature, the second prompt instruction is generated.
The method according to claim 1, wherein the prompt information includes at least one of the following:

Voice prompt information, image prompt information and text prompt information.
The method according to claim 5, wherein said obtaining said response instruction in response to said prompt information comprises:

Acquiring the response instruction issued by the user controlling the terminal device according to the prompt information.
The method according to claim 5, wherein said obtaining said response instruction in response to said prompt information comprises:

Acquiring the second body feature of the user in response to the prompt information;

The corresponding response instruction is generated according to the second limb feature.
A photographing control device, wherein the photographing control device includes a camera module and a vision chip communicatively connected with the camera module, wherein the vision chip includes:

The feature recognition module is used to recognize the user's first body feature;

An instruction generation module, configured to generate a corresponding prompt instruction according to the first limb feature;

An instruction sending module, configured to send the prompt instruction to a terminal device communicatively connected with the shooting control apparatus, so that the terminal device generates corresponding prompt information according to the prompt instruction;

An instruction acquisition module for acquiring a response instruction in response to the prompt information; and

The control module is configured to control the camera module to perform a corresponding camera operation according to the response instruction.
An aircraft comprising a fuselage, an arm connected to the fuselage, a power device arranged on the arm, a camera device connected to the fuselage, and a vision chip arranged on the fuselage , Characterized in that, the vision chip includes: a memory and a processor;

The memory is used to store a computer-executable shooting control program;

The processor is used to call the computer-executable shooting control program to implement the shooting control method according to any one of claims 1-7.
A flight system, the flight system includes an aircraft and a terminal device communicatively connected with the aircraft, the aircraft includes a fuselage, an arm connected to the fuselage, a power device provided on the arm, and The camera device connected to the fuselage and the vision chip provided on the fuselage are characterized in that the vision chip includes:

Memory and processor;

The memory is used to store a computer-executable shooting control program;

The processor is used to call the computer-executable shooting control program to implement the shooting control method according to any one of claims 1-7.