WO2022094808A1

WO2022094808A1 - Photographing control method and apparatus, unmanned aerial vehicle, device, and readable storage medium

Info

Publication number: WO2022094808A1
Application number: PCT/CN2020/126560
Authority: WO
Inventors: 彭阳
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-11-04
Filing date: 2020-11-04
Publication date: 2022-05-12

Abstract

A photographing control method and apparatus, an unmanned aerial vehicle, a device, and a readable storage medium. The method comprises: obtaining a first video (S101); obtaining a second video (S102); and synthesizing the first video and the second video to obtain a target video, the target video comprising at least a part of video frames in the first video and at least a part of video frames in the second video (S103). The present application improves the interestingness and shock of a video.

Description

Shooting control method, device, drone, device and readable storage medium

technical field

The present application relates to the technical field of photographing control, and in particular, to a photographing control method, device, drone, device, and readable storage medium.

Background technique

At present, with the rapid development of Internet technology, more and more users like to share their videos on social platforms. In order to shoot interesting videos to attract the attention of netizens, some users will use drones to shoot videos. Shooting videos from the perspective of drones enriches the diversity of shooting videos and improves the fun of video works to a certain extent, but it is impossible to easily shoot more gorgeous, shocking and interesting videos.

SUMMARY OF THE INVENTION

Based on this, the present application provides a shooting control method, device, unmanned aerial vehicle, device and readable storage medium, aiming at shooting multi-view video, so as to improve the interestingness and shock of the video.

In a first aspect, an embodiment of the present application provides a shooting control method, which is applied to a control device, the control device is used to control a drone and a terminal device, the drone includes a first shooting device, and the terminal device A second photographing device is included, and the method includes:

acquiring a first video, where the first video is obtained by the control device controlling the first shooting device to shoot the user holding the terminal device according to a shooting control instruction triggered by the user;

acquiring a second video, where the second video is obtained by the control device controlling the second shooting device to shoot the drone according to the shooting control instruction;

The first video and the second video are synthesized to obtain a target video, where the target video includes at least a part of video frames in the first video and at least a part of video frames in the second video.

In a second aspect, an embodiment of the present application further provides a control device, the control device is used to control a drone and a terminal device, the drone includes a first photographing device, and the terminal device includes a second photographing device , the control device includes a memory and a processor;

the memory is used to store computer programs;

The processor is configured to execute the computer program and implement the following steps when executing the computer program:

In a third aspect, an embodiment of the present application also provides an unmanned aerial vehicle, including:

body;

a power system, arranged on the body, for providing flight power for the drone;

a first photographing device, mounted on the body, for photographing a first video;

a first wireless communication device, provided on the body, for communicating with the second wireless communication device in the terminal device;

The above control device is used to control the first camera to shoot the first video and/or the drone to fly.

In a fourth aspect, an embodiment of the present application further provides a terminal device, including:

a second shooting device for shooting a second video;

A display device for displaying images or a human-computer interface;

a second wireless communication device for communicating with the first wireless communication device in the drone;

The above control device is used to control the second shooting device to shoot a second video.

In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the above-mentioned The steps of the shooting control method.

Embodiments of the present application provide a photographing control method, device, drone, device, and readable storage medium, by acquiring a first photograph obtained by photographing a user holding a terminal device by a first photographing device in the drone. The video and the second video obtained by photographing the drone by the second shooting device in the terminal device, and synthesizing the first video and the second video to obtain at least a part of the video frame and the second video including the first video. At least a part of the target video in the video frame, so that the target video obtained by synthesis is a multi-view video, which greatly improves the interestingness and shock of the video.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not limiting of the present application.

Description of drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

1 is a schematic diagram of a scene for implementing a shooting control method provided by an embodiment of the present application;

2 is a schematic flowchart of steps of a shooting control method provided by an embodiment of the present application;

3 is a schematic diagram of a scene in which a first video and a second video are synthesized in an embodiment of the present application;

FIG. 4 is a schematic diagram of another scene in which the first video and the second video are synthesized in an embodiment of the present application

5 is a schematic flowchart of steps of another shooting control method provided by an embodiment of the present application;

6 is a schematic diagram of a scenario in which a drone is controlled to fly around and follow a user holding a terminal device in an embodiment of the present application;

7 is a schematic diagram of another scenario in which a drone is controlled to fly around and follow a user holding a terminal device in an embodiment of the present application;

FIG. 8 is a schematic block diagram of the structure of a control device provided by an embodiment of the present application;

9 is a schematic block diagram of the structure of an unmanned aerial vehicle provided by an embodiment of the present application;

FIG. 10 is a schematic structural block diagram of a terminal device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

The flowcharts shown in the figures are for illustration only, and do not necessarily include all contents and operations/steps, nor do they have to be performed in the order described. For example, some operations/steps can also be decomposed, combined or partially combined, so the actual execution order may be changed according to the actual situation.

Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and features in the embodiments may be combined with each other without conflict.

In order to solve the above problems, the embodiments of the present application provide a shooting control method, device, unmanned aerial vehicle, equipment, and readable storage medium. The first video obtained by shooting and the second video obtained by shooting the drone by the second shooting device in the terminal device, and the first video and the second video are synthesized to obtain at least a part of the first video. frame and the target video of at least a part of the video frames in the second video, so that the target video obtained by synthesis is a multi-view video, which greatly improves the interestingness and shock of the video.

Please refer to FIG. 1 . FIG. 1 is a schematic diagram of a scene for implementing the shooting control method provided by the embodiment of the present application. As shown in FIG. 1 , the scene includes an unmanned aerial vehicle 100 and a terminal device 200. The unmanned aerial vehicle 100 includes a body 110, a power system 120 arranged on the body 110, a first photographing device 130 arranged on the body 110, and a A first wireless communication device (not shown in FIG. 1 ) in the body 110, the terminal device 200 includes a display device 210, a second photographing device (not shown in FIG. 1 ) and a second wireless communication device (not shown in FIG. 1 ) out), the first wireless communication device in the drone 100 can communicate with the second wireless communication device in the terminal device 200 , thereby establishing a communication connection between the drone 100 and the terminal device 200 .

Among them, the power system 120 is used to provide the flying power for the drone 100, and the power system 120 can make the drone take off from the ground vertically, or land on the ground vertically, without any horizontal movement of the drone (such as no taxiing on the runway is required). Optionally, the power system 120 may allow the drone to preset positions and/or turn the steering wheel in the air. One or more of the powertrains 120 may be controlled independently of the other powertrains 120 . Alternatively, one or more power systems 120 may be controlled simultaneously. For example, the drone may have multiple horizontally oriented power systems 120 to track the lift and/or push of the target. The horizontally oriented power system 120 may be actuated to provide the drone with the ability to take off vertically, land vertically, and hover.

In one embodiment, one or more of the horizontally oriented power systems 120 may rotate in a clockwise direction, while one or more of the other horizontally oriented power systems may rotate in a counter-clockwise direction. For example, there are as many power systems 120 rotating clockwise as there are power systems 120 rotating counterclockwise. The rotational rate of each power system 120 in the horizontal direction can be varied independently to achieve lift and/or push operations caused by each power system 120 to adjust the spatial orientation, speed and/or acceleration of the drone (eg, relative to multiple rotation and translation up to three degrees of freedom).

In one embodiment, the drone 100 may further include a pan/tilt 140 disposed on the body 110, the pan/tilt 140 is used for carrying the first photographing device 130, and the pan/tilt 140 includes three-axis motors, which are a pan-axis motor, a pitch-axis motor, and a tilt-axis motor. The axis motor and the roll axis motor are used to adjust the balance posture of the first photographing device 130 mounted on the gimbal 140, so as to photograph a high-precision and stable image anytime and anywhere. It can be understood that the terminal device can be a headset equipped with a second camera, AR or VR glasses, a smartphone, a tablet computer, a handheld gimbal and other handheld devices, and the terminal device is a handheld device equipped with a second camera. The gimbal and the hand-held gimbal include a handle and a gimbal for carrying the second shooting device. Through the gimbal, the balance posture of the second shooting device mounted on the gimbal can be adjusted, so that high-precision and stable shooting can be obtained anytime, anywhere. screen.

In one embodiment, the drone may also include a sensing system, which may include one or more sensors to sense the spatial orientation, velocity, and/or acceleration of the drone (eg, relative to up to three Degree of freedom rotation and translation), angular acceleration, attitude, position (absolute position or relative position), etc. The one or more sensors include GPS sensors, motion sensors, inertial sensors, proximity sensors, or image sensors. Optionally, the sensing system may also be used to collect data on the environment in which the UAV 100 is located, such as climatic conditions, potential obstacles to be approached, locations of geographic features, locations of man-made structures, and the like. In addition, the drone 100 may include a tripod, which is a contact piece between the drone 100 and the ground when the drone lands, and the tripod may be the drone 100 in a flying state (for example, the drone 100 is When cruising), it is put away and put down when it is landing; it can also be fixedly installed on the UAV 100, and it is always put down.

In one embodiment, the drone 100 or the terminal device 200 further includes a control device (not shown in FIG. 1 ). When the control device acquires the shooting control instruction triggered by the user, the control device controls the unmanned aerial vehicle according to the shooting control instruction. The first shooting device 130 in the drone 100 shoots the user holding the terminal device 200 to obtain a first video, and at the same time controls the second shooting device in the terminal device 200 to shoot the drone 100 according to the shooting control instruction, The second video is obtained, and then the first video and the second video are synthesized to obtain a target video including at least a part of video frames in the first video and at least a part of video frames in the second video.

In one embodiment, if the drone 100 includes a control device, the control device acquires a user-triggered shooting control instruction sent by the terminal device 200, and controls the first shooting device 130 in the drone 100 according to the shooting control instruction. Shoot the user holding the terminal device 200 to obtain the first video; obtain the second video sent by the terminal device 200, wherein the second video is the terminal device 200 controlling the second shooting device to shoot the drone 100 according to the shooting control instruction. and then synthesizing the first video and the second video to obtain a target video including at least a part of the video frames in the first video and at least a part of the video frames in the second video.

In one embodiment, if the terminal device 200 includes a control device (not shown in FIG. 1 ), the control device controls the terminal device 200 to send the shooting control command to the drone 100 when acquiring the shooting control command triggered by the user, The drone 100 controls the first shooting device to shoot the user holding the terminal device 200 according to the shooting control instruction to obtain the first video, and simultaneously controls the second shooting device to shoot the drone 100 to obtain the second video; Then, the first video sent by the drone 100 is acquired, and the first video and the second video are synthesized to obtain a target video including at least a part of video frames in the first video and at least a part of video frames in the second video.

In one embodiment, the central control device is communicatively connected to the terminal device 200, and the central control device is also communicatively connected to the drone 100. When the terminal device 200 obtains the shooting control instruction triggered by the user, the terminal device 200 sends the drone to the drone. 100 sends a shooting control command, and the terminal device 200 controls the second shooting device to shoot the drone according to the shooting control command, obtains a second video, and sends the second video to the central control device; the drone 100 is receiving When the shooting control instruction arrives, according to the shooting control instruction, the first shooting device 130 is controlled to shoot the user of the handheld terminal device 200 to obtain the first video, and the first video is sent to the central control device; The first video sent by the terminal device 100 and the second video sent by the terminal device 200 are combined, and the first video and the second video are synthesized to obtain at least a part of the video frames in the first video and at least a part of the video frames in the second video. target video.

Hereinafter, the shooting control method provided by the embodiments of the present application will be described in detail with reference to the scene in FIG. 1 . It should be noted that the scene in FIG. 1 is only used to explain the shooting control method provided by the embodiment of the present application, but does not constitute a limitation on the application scene of the shooting control method provided by the embodiment of the present application.

Please refer to FIG. 2 , which is a schematic flowchart of steps of a shooting control method provided by an embodiment of the present application. The photographing control method can be applied to a control device, and the control device is used to control a drone and a terminal device, the drone includes a first photographing device, and the terminal device includes a second photographing device.

As shown in FIG. 2 , the photographing control method includes steps S101 to S103.

Step S101, acquiring a first video, the first video is obtained by the control device controlling the first shooting device to shoot the user holding the terminal device according to a shooting control instruction triggered by the user;

Step S102, acquiring a second video, where the second video is obtained by the control device controlling the second shooting device to shoot the drone according to the shooting control instruction;

Step S103: Synthesize the first video and the second video to obtain a target video, where the target video includes at least a part of video frames in the first video and at least a part of video frames in the second video.

Wherein, the control device may be deployed in the drone, may also be deployed in the terminal device, or may be deployed in the central control device that is communicatively connected with the terminal device and the drone, which is not specifically limited in this embodiment of the present application. The first video may be the current video obtained by the control device controlling the first shooting device to shoot the user holding the terminal device in real time according to the shooting control instruction triggered by the user, or it may be the current video that previously controlled the first shooting device according to the shooting control instruction triggered by the user. A historical video obtained by a shooting device shooting a user holding a terminal device. Similarly, the second video can be obtained by the control device controlling the second shooting device to shoot the drone in real time according to the shooting control instruction triggered by the user. The current video may also be a historical video previously obtained by controlling the second photographing device to photograph the drone according to the photographing control instruction triggered by the user.

In one embodiment, the start and end shooting moments of the first video and the second video are the same, which can ensure the time synchronization of the first video and the second video, and facilitate subsequent synthesis of the first video and the second video. Wherein, in the process of photographing the drone and the user holding the terminal equipment, the user holding the terminal equipment is in the shooting area of the first shooting device of the drone, and the drone is in the second shooting area of the terminal equipment. Within the shooting area of the shooting device, it can be ensured that each video frame in the first video shot by the first shooting device includes the user holding the terminal device, and it can also be ensured that each video in the second video shot by the second shooting device can be guaranteed. All frames include drones, which is convenient for subsequent synthesis to obtain multi-view target videos.

In one embodiment, the first video includes a plurality of first video frames, each of the first video frames includes a user holding a terminal device, and the user holding the terminal device in the first video frame is located in the first video frame. In a preset position, the second video includes a plurality of second video frames, and each second video frame includes a drone. The preset position may be the center position of the first video frame, or may be the remaining positions of the first video frame, which is not specifically limited in this embodiment of the present application.

In one embodiment, the target video includes a plurality of first video frame groups and a plurality of second video frame groups, the first video frame group in the target video is adjacent to the second video frame group, and the first video frame group includes a or more first video frames, and the second video frame group includes one or more second video frames. Exemplarily, extracting a preset number of first video frames from the first video; randomly inserting each first video frame into the second video to obtain the target video; or extracting a preset number of video frames from the second video. Second video frame; randomly insert each second video frame into the first video to obtain the target video. The first video frame in the target video is adjacent to the second video frame, and the preset number may be determined based on the total number of video frames in the first video, which is not specifically limited in this embodiment of the present application.

For example, as shown in FIG. 3 , the first video 11 includes 10 first video frames, and the second video 12 includes 10 second video frames, and 00:01-00:02, 00:00 are extracted from the first video 11 . The first video frame corresponding to 03-00:04, 00:05-00:06, 00:07-00:08 and 00:09-00:10, and 00:01-00:02, 00:03- The first video frames corresponding to 00:04, 00:05-00:06, 00:07-00:08, and 00:09-00:10 are inserted into the second video 12 at 00:01 and 00:03 respectively. , 00:05, 00:07 and 00:09 corresponding positions, thereby obtaining the target video 13, the target video 13 includes 15 video frames, and the video duration is 15 seconds.

Exemplarily, a preset number of first video frames are extracted from the first video, and according to the shooting moment of each first video frame, a matching second video frame is extracted from the second video frame to obtain a preset number of video frames. The second video frame, wherein the matching first video frame and the second video frame have the same shooting time; according to the shooting time of each second video frame, insert each second video frame into the extracted first video frame In the first video, the target video is obtained, or, according to the shooting moment of each first video frame, each first video frame is inserted into the second video after the second video frame is extracted.

For example, as shown in FIG. 4, 00:01-00:02, 00:03-00:04, 00:05-00:06, 00:07-00:08, and 00:09 are extracted from the first video 21 - the first video frame corresponding to 00:10, and extract 00:01-00:02, 00:03-00:04, 00:05-00:06, 00:07-00:08 and For the second video frame corresponding to 00:09-00:10, obtain the to-be-inserted video 22 after extracting the second video frame, and then set the video time as 00:01-00:02, 00:03-00:04, 00: The first video frames of 05-00:06, 00:07-00:08 and 00:09-00:10 are inserted into the video to be inserted 22 to obtain the target video 23 .

In some embodiments, the first video, the second video, and the third video may also be acquired, and the viewing angles for shooting the first video, the second video, and the third video are different; Synthesize to obtain a target video, where the target video includes part or all of the video frames of the first video, part or all of the video frames of the second video, and part or all of the video frames of the third video. It can be understood that more videos with different viewing angles can also be obtained, and more videos with different viewing angles can be synthesized to obtain videos with multiple viewing angles.

The first video is obtained by the control device controlling the first shooting device in the first drone to shoot the user holding the terminal device according to the shooting control instruction triggered by the user, and the second video is obtained by the control device according to the shooting control instruction The third video is obtained by controlling the second shooting device in the terminal device to shoot the drone, and the third video is obtained by the control device controlling the third shooting device in the second drone according to the shooting control instruction triggered by the user, to hold the terminal device. Obtained by the user shooting, and the user holding the terminal device is located in the shooting area of the first shooting device and the third shooting device, and the first drone and/or the second drone is located in the shooting area of the second shooting device .

In one embodiment, the terminal device is controlled to output shooting prompt information, wherein the shooting prompt information is used to prompt the user to shoot a video, and the shooting prompt information is also used to prompt the user that the drone is already in the shooting area of the second shooting device, And the user holding the terminal device is also in the shooting area of the first shooting device; the shooting control instruction triggered by the user is obtained, and according to the shooting control instruction, the first shooting device is controlled to shoot the user holding the terminal device, and the first shooting control instruction is obtained. video, and according to the shooting control instruction, the second shooting device is controlled to shoot the drone to obtain a second video; the first video and the second video are synthesized to obtain a target video, wherein the target video includes the first video at least a portion of the video frames in the second video and at least a portion of the video frames in the second video. When the user's drone is already in the shooting area of the second shooting device, and the user holding the terminal device is also in the shooting area of the first shooting device, the user is prompted to shoot video, which facilitates the user to control the first shooting device and The second shooting device shoots the video synchronously to ensure that the time of the captured video is strictly synchronized.

In one embodiment, before the terminal device is controlled to output the shooting prompt information, the terminal device is controlled to display a shooting control page, wherein the shooting control page includes a viewing angle switching icon and the first image collected by the first shooting device; The user's touch operation in the first image takes the user in the first image as the first photographing subject of the first photographing device; in response to the user's triggering operation on the viewing angle switching icon, the photographing of the first image in the control page is performed. switching to the second image collected by the second photographing device; in response to the user's triggering operation on the drone in the second image, the drone in the second image is used as the second photographing subject of the second photographing device; When it is determined that the first shooting subject of the first shooting device is the user holding the terminal device, and the second shooting subject of the second shooting device is the drone, the terminal device is controlled to output shooting prompt information to remind the user that the drone is in the Within the shooting area of the second shooting device, and the user holding the terminal device is also within the shooting area of the first shooting device.

In an embodiment, when there is no user holding the terminal device in the first image, the terminal device is controlled to output first prompt information, and the first prompt information is used to prompt the user to adjust the first shooting area of the first shooting device; according to The first posture adjustment instruction triggered by the user adjusts the posture of the first photographing device, and the first photographing area and the displayed first image change as the posture of the first photographing device changes. Wherein, the posture of the first photographing device may be adjusted by adjusting the posture of the drone, or the posture of the first photographing device may be adjusted by adjusting the posture of the pan/tilt on which the first photographing device is mounted. By outputting prompt information when the user holding the terminal device is not located in the shooting area of the first shooting device, it is convenient for the user to adjust the posture of the first shooting device and the user experience is improved.

In an embodiment, the manner of controlling the terminal device to output the first prompt information may be: acquiring first orientation information of the drone relative to the terminal device; and controlling the terminal device to output the first prompt information according to the first orientation information. The first prompt information includes first orientation information of the drone relative to the terminal device. By outputting the prompt information including the orientation information of the drone relative to the terminal device, it is convenient for the user to adjust the posture of the first photographing device based on the orientation information in the prompt information, reducing the number of times the user adjusts the posture of the first photographing device, and improving the user experience. experience.

In one embodiment, the method of acquiring the first position information of the drone relative to the terminal device may be: acquiring the first position information of the drone and the second position information of the terminal device; according to the first position information and the second position information position information, and determine the first orientation information of the drone relative to the terminal device. Or obtain the first position information of the drone, the second position information of the terminal device and the direction information output by the compass sensor of the terminal device; according to the first position information, the second position information and the direction information, determine the relative position of the drone to the terminal The first orientation information of the device.

In an embodiment, when there is a user holding a terminal device in the updated first image, second prompt information is output, wherein the second prompt information is used to prompt the user to stop adjusting the first shooting area, and is also used to prompt the user to stop adjusting the first shooting area. The user holding the terminal device is located within the shooting area of the first shooting device. When there is a user holding a terminal device in the first image captured after adjusting the posture of the first photographing device, prompt information for prompting the user to stop adjusting the posture of the first photographing device is output, so that the user knows that it is not necessary to further adjust the posture of the first photographing device. Adjusting the posture of the first photographing device greatly improves the user experience.

In one embodiment, when there is no drone in the second image, the terminal device is controlled to output third prompt information, wherein the third prompt information is used to prompt the user to adjust the second shooting area of the second shooting device; according to the user The triggered second posture adjustment instruction adjusts the posture of the second photographing device, wherein the second photographing area and the displayed second image change as the posture of the second photographing device changes. By outputting prompt information when there is no drone in the second image, it is convenient for the user to adjust the posture of the second photographing device, and the user experience is improved.

In one embodiment, controlling the terminal device to output the third prompt information includes: acquiring second orientation information of the terminal device relative to the drone; and controlling the terminal device to output the third prompt information according to the second orientation information. The second prompt information includes second orientation information of the terminal device relative to the drone. By outputting prompt information including the orientation information of the terminal device relative to the drone, it is convenient for the user to adjust the attitude of the second photographing device based on the orientation information in the prompt information, reducing the number of times the user adjusts the attitude of the second photographing device, and improving the user experience. experience.

In one embodiment, the manner of acquiring the second orientation information of the terminal device relative to the drone may be: acquiring the first location information of the drone and the second location information of the terminal device; Position information, to determine the second orientation information of the terminal device relative to the drone. Or obtain the first location information of the drone, the second location information of the terminal device, and the direction information output by the compass sensor of the terminal device; The second orientation information of the machine.

In one embodiment, when there is a drone in the updated second image, fourth prompt information is output, wherein the fourth prompt information is used to prompt the user to stop adjusting the second shooting area, and is also used to prompt the user that no one is there. The camera is located in the shooting area of the second shooting device. By outputting prompt information for prompting the user to stop adjusting the posture of the second photographing device when there is a drone in the second image captured after adjusting the posture of the second photographing device, the user knows that it is not necessary to adjust the second photographing device. The posture of the shooting device greatly improves the user experience.

In one embodiment, the shooting control page also displays a flight shooting mode option of the drone, the user can select the flight shooting mode of the drone through the flight shooting mode option, and the flight shooting mode of the drone includes a return shooting mode. , away from shooting mode, normal follow mode and surround follow mode, when the first shooting device is in the return shooting mode, the user is prompted to maintain the attitude of the second shooting device unchanged, and the control device can control the drone from the set position to the holding position. The position of the user with the terminal equipment is to fly close to the user holding the terminal equipment, and hover when the height of the drone is lower than the set height; when the first shooting device is in the remote shooting mode, the user is prompted Keeping the attitude of the second shooting device unchanged, the control device can control the drone to fly from the position of the user holding the terminal device to the set position; when the drone is in the normal follow mode, the control device can The movement speed and/or movement trajectory of the user of the terminal device follows the user holding the terminal device; when the drone is in the follow-around mode, the control device can control the drone to surround the drone according to the rotation speed of the user holding the terminal device. Fly with the user holding the terminal device.

The shooting control method provided by the above-mentioned embodiment, by acquiring the first video obtained by the first shooting device in the drone shooting the user holding the terminal device, and the second shooting device in the terminal device shooting the drone. Shooting the obtained second video, and synthesizing the first video and the second video to obtain a target video comprising at least a part of video frames in the first video and at least a part of video frames in the second video, so that the target video obtained by synthesis For multi-view video, it greatly improves the fun and shock of the video.

Please refer to FIG. 5 , which is a schematic flowchart of steps of another shooting control method provided by an embodiment of the present application.

Specifically, as shown in FIG. 5 , the shooting control method includes steps S201 to S202.

S201, in the process of controlling the first photographing device to photograph the user holding the terminal device, acquire the flight trajectory in the photographing control instruction;

S202. Control the drone to fly according to the flight trajectory, so that the drone is far away from the user holding the terminal device or close to the user holding the terminal device.

In the process of controlling the first photographing device to photograph the user holding the terminal device, the mode identifier in the photographing control instruction is acquired, and the mode identifier is the first mode identifier corresponding to the return photographing mode or the first mode identifier corresponding to the remote photographing mode. When the second mode is marked, the control device obtains the flight trajectory in the shooting control command, and controls the drone to fly according to the flight trajectory, so that the drone is far away from the user holding the terminal device or close to the user holding the terminal device, so that the The first photographing device can photograph a video with a viewing angle far away from the user or close to the user. Wherein, the flight trajectory includes any one of the first flight trajectory and the second flight trajectory, so the drone can be controlled to fly according to the first flight trajectory, so that the drone is far away from the user holding the terminal device; The man-machine flies according to the second flight trajectory, so that the drone is close to the user holding the terminal device.

The mode identification in the shooting control instruction is used to indicate the flight shooting mode of the drone, and the mode identifications of different flight shooting modes are different. For example, the mode identification includes a first mode identification, a second mode identification, a third mode identification and The fourth mode identification, and the first mode identification corresponds to the return shooting mode, the second mode identification corresponds to the remote shooting mode, the third mode identification corresponds to the normal follow mode, and the fourth mode identification corresponds to the surround follow mode.

The first flight trajectory includes a first starting waypoint and a first ending waypoint. When the drone is at the first starting waypoint, the hovering height of the drone is within a preset height range, and the drone is connected to the holding terminal. The horizontal distance between users of the device is within the preset distance range, so that the drone and the user holding the terminal device maintain a certain range to ensure the flight safety of the drone. The second flight trajectory includes the second starting waypoint and the second ending waypoint. When the drone is at the second ending waypoint, the hovering height of the drone is within the preset height range, and the drone is connected to the terminal device held by the drone. The horizontal distance between the users of the device is within the preset distance range, so that the drone and the user holding the terminal device maintain a certain range to ensure the flight safety of the drone. The preset height range and the preset distance range may be set based on actual conditions, which are not specifically limited in this embodiment of the present application. For example, the preset height range is 2 meters to 3 meters, and the preset distance range is 1 meter to 2 meters. .

In one embodiment, in the process of controlling the first photographing apparatus to photograph the user holding the terminal device, the drone is controlled to follow the user holding the terminal device to fly. Further, in the process of controlling the first shooting device to shoot the user holding the terminal device, the mode identification in the shooting control instruction is obtained, and the mode identification is the third mode identification corresponding to the ordinary following mode or the surround following mode. When the corresponding fourth mode is identified, the drone is controlled to follow the user holding the terminal device to fly. By controlling the drone to follow the user holding the terminal device to fly, the first photographing device can shoot a video with a viewing angle of following the user holding the terminal device.

In an embodiment, the method of controlling the drone to follow the user holding the terminal device to fly may be: acquiring the current image collected by the first photographing device; controlling the drone to follow the user holding the terminal device to fly according to the current image. The method of controlling the drone to follow the user holding the terminal device according to the current image collected by the first photographing device may be: inputting the current image collected by the first photographing device into the preset target detection model, so as to detect the first A user in a current image collected by a photographing device performs target detection, and obtains target detection information of the user, wherein the target detection information includes the size information and position information of the user in the world coordinate system and the relative drone of the user. according to the target detection information and the preset target tracking algorithm, predict the target position information of the user holding the terminal device at the next moment; according to the target position information and the current position information of the UAV, control the UAV to follow The user holding the terminal device flies.

Wherein, the size information of the user holding the terminal device in the world coordinate system includes the length information, width information and/or height information of the user holding the terminal device, and the angle information of the user holding the terminal device relative to the drone includes: The yaw angle, pitch angle and roll angle of the user holding the terminal device relative to the drone. The preset target detection model includes a 2D target detection model or a 3D target detection model. The 3D target detection model is the pre-trained first neural network. Model, the 2D target detection model is a pre-trained second neural network model. The first neural network model is different from the second neural network model. The first neural network model includes convolutional neural network models CNN, RCNN, Fast RCNN and Faster RCNN Any one of the second neural network model includes any one of the convolutional neural network model CNN, RCNN, Fast RCNN and Faster RCNN.

In one embodiment, the method of training the first neural network model to obtain the 3D target detection model may be: acquiring first training sample data, wherein the first training sample data includes a plurality of first images and each first image The first target detection information of the target to be tracked in ; the first neural network model is iteratively trained according to the first training sample data, until the iteratively trained first neural network model converges, and a 3D target detection model is obtained. By including the yaw angle, pitch angle and roll angle of the target to be tracked relative to the UAV, the position information of the target to be tracked in the camera coordinate system, and the first size information of the target to be tracked in the world coordinate system, etc. The detection information and corresponding images are used to train the first neural network model, which can solve the problem that the existing 3D target detection algorithm cannot be reused on the UAV, so that the UAV can detect the target to be tracked based on the 3D target detection model. , which is convenient for subsequent and accurate control of the drone to follow the user holding the terminal device to fly.

In one embodiment, the method of training the second neural network model to obtain the 2D target detection model may be: acquiring second training sample data, wherein the second training sample data includes a plurality of second images and each The second target detection information of the target to be tracked in the two images; the second neural network model is iteratively trained according to the second training sample data, until the iteratively trained second neural network model converges, and a 2D target detection model is obtained. By training the second neural network model with the second target detection information including the position information and the second size information of the target to be tracked in the current captured image and the corresponding image, a 2D target detection model can be obtained, so that the UAV can Target detection can be performed on the target to be tracked based on the 2D target detection model, which is convenient for subsequent and accurate control of the drone to follow the user holding the terminal device to fly.

The preset target tracking algorithm includes any one of a mean shift algorithm, a Kalman filter algorithm, a particle filter algorithm, and a moving target modeling algorithm. In some other embodiments, other target tracking algorithms may also be used, which are not limited herein.

In one embodiment, the method of controlling the drone to follow the user holding the terminal device to fly may be: obtaining the target rotation speed of the user holding the terminal device; according to the target rotation speed, controlling the drone to follow the terminal device around the drone of users flying. Wherein, in the process of controlling the drone to fly around and follow the user holding the terminal device, the user holding the terminal device is always in the shooting area of the first shooting device, and the drone is always in the shooting area of the second shooting device . Through the target rotation speed of the user holding the terminal device, the drone is controlled to fly around and follow the user holding the terminal device, so that the first shooting device can shoot a more splendid video.

Exemplarily, as shown in FIG. 6 , the user 31 holding the terminal device rotates on the spot according to the rotation track 32, and the rotation direction is counterclockwise, and the shooting area of the second shooting device on the terminal device is the line 33 and the line 32. In the area between 34, the drone 35 is located in the area between the line 33 and the line 34. The drone 35 flies around and follows the user holding the terminal device 31 according to the follow-around track 37, and the surrounding direction is also counterclockwise. The photographing area of the first photographing device on the drone 35 is the area between the

lines

37 and 38 , and the user 31 holding the terminal device is located in the area between the

lines

37 and 38 .

In one embodiment, according to the target rotation speed, the method of controlling the drone to fly around and follow the user holding the terminal device may be: acquiring the starting waypoint of the surround in the shooting control instruction, and controlling the drone to fly towards the surround. Flying at the starting point; after the drone reaches the starting waypoint and flying around the starting point, according to the target rotation speed of the user holding the terminal device, the drone is controlled to start from the starting point of User flies. Wherein, the encircling starting waypoint in the shooting control instruction may be input by the user through the shooting control page of the terminal device.

In an embodiment, the terminal device includes an inertial measurement unit and a compass sensor, and the method of acquiring the target rotation speed of the user holding the terminal device may be: acquiring the first sensing data collected by the inertial measurement unit, 1. sensing data, to determine the target rotation speed of the user holding the terminal device; or, to obtain the first sensing data collected by the inertial measurement unit and the second sensing data collected by the compass sensor; according to the first sensing data data and the second sensory data to determine the target rotational speed of the user holding the terminal device. The target rotation speed of the user holding the terminal device can be accurately determined by the inertial measurement unit, so that the drone can fly around and follow the user accurately according to the target rotation speed.

In an embodiment, the method of acquiring the target rotation speed of the user holding the terminal device may be: acquiring images collected by the first photographing device at different times, and determining that the identification target set on the terminal device is in each image. According to the pixel coordinates of the identification target in each image, the target rotation speed of the user holding the terminal device is determined. Wherein, the identification target includes an anti-reflection film set on the terminal device, and the anti-reflection film is more easily recognized by the first photographing device. The rotation speed of the marking target set on the terminal equipment can be determined through the images collected by the first photographing device at different times. Since the marking target is set on the terminal equipment, the rotation speed of the marking target can be determined as holding the terminal equipment. The user's target rotation speed.

In an embodiment, the terminal device includes an inertial measurement unit and a compass sensor, and the method of acquiring the target rotation speed of the user holding the terminal device may be: acquiring the first sensor data collected by the inertial measurement unit and setting it on the terminal device. The target rotation speed of the user holding the terminal device is determined according to the first sensing data and the rotation speed of the identified target. By comprehensively determining the target rotation speed of the user holding the terminal device according to the first sensing data and the rotation speed of the identified target, the accuracy of the target rotation speed of the user holding the terminal device can be improved.

In an embodiment, according to the first sensing data and the rotation speed of the identified target, the method of determining the target rotation speed of the user holding the terminal device may be: according to the first sensing data, determining the rotation speed of the user holding the terminal device. The first rotation speed, and according to the first rotation speed, the first preset weight coefficient corresponding to the first rotation speed, the rotation speed of the identification target, and the second preset weight coefficient corresponding to the rotation speed of the identification target, determine the holding terminal device The user's target rotation speed. The first preset weight coefficient and the second preset weight coefficient may be set based on actual conditions, which are not specifically limited in this embodiment of the present application. For example, the first preset weight coefficient is 0.7, and the second preset weight coefficient is 0.7. is 0.3. For example, if the first rotation speed is w ₁ , the first preset weight coefficient is k ₁ , the rotation speed of the identification target is w ₂ , and the second preset weight coefficient is k ₂ , the target rotation speed may be w ₁ *k ₁ +w ₂ *k ₂ . By setting different weight coefficients for the rotation speeds determined in different ways to determine the target rotation speed of the user holding the terminal device, the accuracy of the target rotation speed of the user holding the terminal device can be improved.

In one embodiment, the terminal device includes an inertial measurement unit and a compass sensor, and the method of acquiring the target rotation speed of the user holding the terminal device may be: acquiring the first sensing data collected by the inertial measurement unit, the The second sensory data and the rotation speed of the identification target set on the terminal device; according to the first sensory data, the second sensory data and the rotation speed of the identification target, determine the target rotation speed of the user holding the terminal device . By comprehensively considering the data collected by the inertial measurement unit, the data collected by the compass sensor and the rotation speed of the identified target to determine the target rotation speed, the accuracy of the target rotation speed can be further improved.

In an embodiment, according to the first sensing data, the second sensing data and the rotation speed of the identified target, the method of determining the target rotation speed of the user holding the terminal device may be: according to the first sensing data and the The second sensing data determines the second rotation speed of the user holding the terminal device; according to the second rotation speed, the third preset weight coefficient corresponding to the second rotation speed, the rotation speed of the identification target, and the rotation speed of the identification target corresponding to The second preset weight coefficient determines the target rotation speed of the user holding the terminal device. For example, if the second rotation speed is w ₃ , the third preset weight coefficient is k ₃ , the rotation speed of the identification target is w ₂ , and the second preset weight coefficient is k ₂ , the target rotation speed may be w ₃ *k ₃ +w ₂ *k ₂ . By setting different weight coefficients for the rotation speeds determined in different ways to determine the target rotation speed of the user holding the terminal device, the accuracy of the target rotation speed of the user holding the terminal device can be improved.

In one embodiment, according to the target rotation speed, the method of controlling the drone to fly around and follow the user holding the terminal device may be: obtaining the surrounding flight angle of the drone; controlling the drone to follow and hold the drone according to the target rotation speed The user of the terminal device flies until the angle at which the drone flies around the user reaches a circling flight angle. The surrounding flight angle can be set by the user through the shooting control page of the terminal device, or can be extracted and preset, which is not specifically limited in this embodiment of the present application. For example, the surrounding flight angle includes 90°, 120°, 180°, 240°, 270°, 300°, 360°, etc.

In one embodiment, in the process of controlling the drone to fly around and follow the user, if a throwing control command sent by the terminal device is detected, the throwing direction of the drone is determined; the drone is controlled along the throwing direction. flight. The method of determining the flying direction of the UAV may be as follows: if the flying control command sent by the terminal device is detected, the flying speed of the UAV at the current position is determined, and the direction of the horizontal component of the flying speed is determined. , and the direction of the horizontal component of the flight speed is determined as the flying direction of the UAV. The terminal device includes a flight trigger control key, and when a user triggering operation on the flight trigger control key is detected, a flight control instruction is generated, and the flight trigger control key can be a physical button or a virtual button. This application The embodiment does not specifically limit this. By controlling the drone to fly along the throwing direction, the first photographing device can shoot a video with the throwing effect.

In one embodiment, exemplarily, as shown in FIG. 7 , the user 41 holding the terminal device rotates on the spot according to the rotation trajectory 42 , and the rotation direction is counterclockwise, and the drone starts around the starting waypoint 43 Follow the user 41 holding the terminal device to fly around according to the follow-around trajectory 44, and the surrounding direction is also counterclockwise. When detecting the flight control command sent by the terminal device, the drone flies around to the waypoint 44. At this time, there is no The man-machine flies along the direction of the throwing trajectory 45 from the waypoint 44 , and hovers when the drone reaches the waypoint 46 .

In one embodiment, the manner of controlling the drone to fly along the throwing direction may be: controlling the drone to fly along the throwing direction until the distance of the drone flying along the throwing direction reaches a preset distance; or Control the drone to fly along the throwing direction until the flying distance of the drone along the throwing direction reaches the throwing distance in the throwing control command; or, control the drone to decelerate and fly along the throwing direction until The horizontal flight speed of the drone becomes zero. Wherein, in the process of controlling the drone to fly along the throwing direction, the posture of the first photographing device is adjusted so that the user holding the terminal device is always within the photographing area of the first photographing device.

In one embodiment, in the process of controlling the drone to fly around and follow the user holding the terminal device, the posture of the first photographing device is adjusted so that the holding terminal in the image collected by the first photographing device after the posture is adjusted. The user of the device is in a preset position in this image. Wherein, the preset position may be the central position of the image, or may be other positions in the image, which is not specifically limited in this application. By adjusting the posture of the first photographing device so that the user holding the terminal device in the image captured by the first photographing device after the posture is adjusted is located at a preset position in the image, the photographing subject of the first photographing device can always be Users with terminal equipment can improve the shooting effect.

In one embodiment, in the process of controlling the drone to fly around and follow the user holding the terminal device, the image captured by the first photographing device is acquired, and it is determined whether the position of the user holding the terminal device in the image is The preset position; if the position of the user holding the terminal device in the image is not the preset position, and the deflection angle of the pan/tilt head equipped with the first photographing device is greater than the preset angle, the terminal device is controlled to output rotation speed adjustment prompt information , to prompt the user to adjust its own rotation speed, to ensure that the drone can better fly around and follow the user holding the terminal device, and to improve the video shooting effect and user experience.

In the shooting control method provided by the above embodiment, in the process of controlling the first shooting device to shoot the user holding the terminal device, the flight trajectory in the shooting control instruction is acquired, and the drone is controlled to fly according to the flight trajectory, The drone is kept away from the user holding the terminal device or close to the user holding the terminal device, so that the first photographing device can shoot a video with a viewing angle far away from the user or close to the user.

Please refer to FIG. 8. FIG. 8 is a schematic structural block diagram of a control apparatus provided by an embodiment of the present application.

As shown in FIG. 8 , the control device 300 includes a processor 301 and a memory 302, and the processor 301 and the memory 302 are connected through a bus 303, such as an I2C (Inter-integrated Circuit) bus. The control device 300 is used to control the drone and the terminal device, the drone includes a first photographing device, the terminal device includes a second photographing device, and the drone is communicatively connected to the terminal device.

Specifically, the processor 301 may be a micro-controller unit (Micro-controller Unit, MCU), a central processing unit (Central Processing Unit, CPU) or a digital signal processor (Digital Signal Processor, DSP) or the like.

Specifically, the memory 302 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) magnetic disk, an optical disk, a U disk, a mobile hard disk, and the like.

Wherein, the processor 301 is used for running the computer program stored in the memory 302, and implements the following steps when executing the computer program:

In an embodiment, the start and end shooting moments of the first video and the second video are the same.

In one embodiment, during the process of photographing the user and the drone, the user is in the photographing area of the first photographing device, while the drone is photographing the second within the shooting area of the device.

In one embodiment, the first video includes a plurality of first video frames, each of the first video frames includes the user, and the user in the first video frame is located in the first video frame. The preset position of the frame, the second video includes a plurality of second video frames, and each of the second video frames includes the drone.

In one embodiment, the target video includes a plurality of first video frame groups and a plurality of second video frame groups, and the first video frame group in the target video is adjacent to the second video frame group , the first video frame group includes one or more first video frames, and the second video frame group includes one or more second video frames.

In one embodiment, the processor is further configured to implement the following steps:

Controlling the terminal device to output shooting prompt information, and acquiring a shooting control instruction triggered by the user, wherein the shooting prompt information is used to prompt the user to shoot a video;

The acquiring the first video includes:

According to the shooting control instruction, controlling the first shooting device to shoot the user holding the terminal device to obtain a first video;

The obtaining of the second video includes:

According to the shooting control instruction, the second shooting device is controlled to shoot the drone to obtain a second video.

In one embodiment, before the processor controls the terminal device to output the shooting prompt information, the processor is further configured to:

controlling the terminal device to display a shooting control page, wherein the shooting control page includes a viewing angle switching icon and a first image collected by the first shooting device;

In response to the user's touch operation on the user in the first image, the user in the first image is used as the first photographing subject of the first photographing device;

In response to the user's triggering operation on the viewing angle switching icon, switching the first image in the shooting control page to the second image collected by the second shooting device;

In response to the user's triggering operation on the drone in the second image, the drone in the second image is used as the second photographing subject of the second photographing device;

When it is determined that the first photographing subject of the first photographing device is the user and the second photographing subject of the second photographing device is the drone, the terminal device is controlled to output photographing prompt information.

When the user does not exist in the first image, controlling the terminal device to output first prompt information, where the first prompt information is used to prompt the user to adjust the first shooting area of the first shooting device;

The posture of the first photographing device is adjusted according to a first posture adjustment instruction triggered by the user, and the first photographing area and the displayed first image change with the posture of the first photographing device.

In an embodiment, the controlling the terminal device to output the first prompt information includes:

acquiring first orientation information of the drone relative to the terminal device;

According to the first orientation information, the terminal device is controlled to output first prompt information.

When the user exists in the updated first image, second prompt information is output, wherein the second prompt information is used to prompt the user to stop adjusting the first shooting area.

When the drone does not exist in the second image, controlling the terminal device to output third prompt information, where the third prompt information is used to prompt the user to adjust the second shooting area of the second shooting device;

The posture of the second photographing device is adjusted according to the second posture adjustment instruction triggered by the user, and the second photographing area and the displayed second image change with the change of the posture of the second photographing device.

In an embodiment, the controlling the terminal device to output the third prompt information includes:

acquiring the second orientation information of the terminal device relative to the drone;

According to the second orientation information, the terminal device is controlled to output third prompt information.

When the drone exists in the updated second image, fourth prompt information is output, wherein the fourth prompt information is used to prompt the user to stop adjusting the second shooting area.

in the process of controlling the first photographing device to photograph the user holding the terminal device, acquiring the flight trajectory in the photographing control instruction;

The drone is controlled to fly according to the flight trajectory, so that the drone is far away from the user holding the terminal device or close to the user holding the terminal device.

In one embodiment, the flight trajectory includes any one of a first flight trajectory and a second flight trajectory, and the controlling the UAV to fly according to the flight trajectory includes:

Controlling the drone to fly according to the first flight trajectory, so that the drone is far away from the user holding the terminal device;

or

The drone is controlled to fly according to the second flight trajectory, so that the drone is close to the user holding the terminal device.

In the process of controlling the first photographing apparatus to photograph the user holding the terminal device, the drone is controlled to follow the user holding the terminal device to fly.

In one embodiment, the controlling the drone to follow the user holding the terminal device to fly includes:

obtaining the current image collected by the first photographing device;

The drone is controlled to follow the user holding the terminal device to fly according to the current image.

In an embodiment, the controlling the drone to follow the user holding the terminal device to fly according to the current image includes:

Inputting the current image into a preset target detection model, to perform target detection on the user in the current image, and obtain target detection information of the user, wherein the target detection information includes the user in the world Size information, position information and angle information of the user relative to the drone in the coordinate system;

According to the size information, the angle information and the preset target tracking algorithm, predict the target position information of the user at the next moment;

According to the target location information and the current location information of the drone, the drone is controlled to follow the user holding the terminal device to fly.

obtaining the target rotation speed of the user holding the terminal device;

According to the target rotation speed, the drone is controlled to fly around and follow the user holding the terminal device.

In one embodiment, the terminal device includes an inertial measurement unit and a compass sensor, and the acquiring the target rotation speed of the user holding the terminal device includes:

acquiring the first sensing data collected by the inertial measurement unit, and determining the target rotation speed of the user holding the terminal device according to the first sensing data;

or

Obtain the first sensing data collected by the inertial measurement unit and the second sensing data collected by the compass sensor;

According to the first sensing data and the second sensing data, a target rotation speed of the user holding the terminal device is determined.

In an embodiment, the acquiring the target rotation speed of the user holding the terminal device includes:

Acquiring images collected by the first photographing device at different moments, and determining the pixel coordinates of the identification target set on the terminal device in each of the images;

According to the pixel coordinates of the identification target in each of the images, the target rotation speed of the user holding the terminal device is determined.

In one embodiment, the identification target includes an antireflection film provided on the terminal device.

Acquiring the first sensing data collected by the inertial measurement unit and the rotation speed of the identified target set on the terminal device;

determining the target rotation speed of the user holding the terminal device according to the first sensing data and the rotation speed of the identified target;

or

Acquiring the first sensing data collected by the inertial measurement unit, the second sensing data collected by the compass sensor, and the rotation speed of the marked target set on the terminal device;

According to the first sensing data, the second sensing data and the rotation speed of the identified target, the target rotation speed of the user holding the terminal device is determined.

In one embodiment, controlling the drone to fly around and following the user holding the terminal device according to the target rotation speed includes:

obtain the surrounding flight angle of the UAV;

The drone is controlled to fly around and follow the user holding the terminal device according to the target rotation speed, until the flying angle of the drone around the user reaches the around flying angle.

In the process of controlling the drone to fly around and follow the user, if a throwing control command sent by the terminal device is detected, determining the throwing direction of the drone;

The drone is controlled to fly along the throwing direction.

In one embodiment, the controlling the drone to fly along the throwing direction includes:

Controlling the drone to fly along the throwing direction, until the distance that the drone flies along the throwing direction reaches a preset distance;

or

Controlling the drone to fly along the throwing direction until the flying distance of the drone along the throwing direction reaches the throwing distance in the throwing control instruction;

or

The UAV is controlled to decelerate along the throwing direction until the horizontal flight speed of the UAV becomes zero.

During the process of controlling the drone to fly around and follow the user, the posture of the first photographing device is adjusted so that the user in the image captured by the first photographing device after the posture is adjusted is located in the Preset positions in the image.

It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the control device described above, reference may be made to the corresponding process in the above-mentioned embodiments of the shooting control method, which is not repeated here. Repeat.

Please refer to FIG. 9. FIG. 9 is a schematic structural block diagram of an unmanned aerial vehicle provided by an embodiment of the present application.

As shown in FIG. 9 , the drone 400 includes: a body 410 , a power system 420 , a first photographing device 430 , a first wireless communication device 440 and a control device 450 , a power system 420 , a first photographing device 430 , and a first wireless communication device The device 440 and the control device 450 are arranged on the body 410, the power system 420 is used to provide flight power for the UAV 400, the first photographing device 430 is used for the first video, and the first wireless communication device 440 is used to communicate with the terminal equipment. The second wireless communication device performs communication, and the control device 450 is configured to control the first shooting device 430 to shoot the first video and/or the drone 400 to fly.

It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of the description, for the specific working process of the UAV described above, reference may be made to the corresponding process in the above-mentioned embodiments of the shooting control method. Repeat.

Please refer to FIG. 10. FIG. 10 is a schematic structural block diagram of a terminal device provided by an embodiment of the present application.

As shown in FIG. 10 , the terminal device 500 includes a second photographing apparatus 510 , a display apparatus 520 , a second wireless communication apparatus 530 and a control apparatus 540 , the second photographing apparatus 510 is used for photographing a second video, and the display apparatus 520 is used for displaying images Or a human-computer interaction interface, the second wireless communication device 530 is used to communicate with the first wireless communication device in the drone, and the control device is used to control the second shooting device to shoot the second video.

It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of the description, for the specific working process of the terminal device described above, reference may be made to the corresponding process in the foregoing embodiment of the shooting control method, which is not repeated here. Repeat.

Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, the computer program includes program instructions, and the processor executes the program instructions, so as to realize the provision of the above embodiments. The steps of the shooting control method.

The computer-readable storage medium may be an internal storage unit of the terminal device or the drone described in any of the foregoing embodiments, such as a hard disk or memory of the terminal device or the drone. The computer-readable storage medium can also be an external storage device of the terminal device or the drone, such as a plug-in hard disk equipped on the terminal device or the drone, a smart memory card (Smart Media Card, SMC), Secure Digital (SD) card, flash memory card (Flash Card), etc.

It should be understood that the terms used in the specification of the present application herein are for the purpose of describing particular embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural unless the context clearly dictates otherwise.

It will also be understood that, as used in this specification and the appended claims, the term "and/or" refers to and including any and all possible combinations of one or more of the associated listed items.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalents within the technical scope disclosed in the present application. Modifications or substitutions shall be covered by the protection scope of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

A shooting control method, characterized in that it is applied to a control device, the control device is used to control a drone and a terminal device, the drone includes a first shooting device, and the terminal device includes a second shooting device, The method includes:

acquiring a first video, where the first video is obtained by the control device controlling the first shooting device to shoot the user holding the terminal device according to a shooting control instruction triggered by the user;

acquiring a second video, where the second video is obtained by the control device controlling the second shooting device to shoot the drone according to the shooting control instruction;

The first video and the second video are synthesized to obtain a target video, where the target video includes at least a part of video frames in the first video and at least a part of video frames in the second video.
The shooting control method according to claim 1, wherein the start and end shooting moments of the first video and the second video are the same.
The shooting control method according to claim 1, wherein in the process of shooting the user and the drone, the user is in the shooting area of the first shooting device, and the The drone is located within the shooting area of the second shooting device.
The shooting control method according to claim 1, wherein the first video includes a plurality of first video frames, each of the first video frames includes the user, and the first video frame includes the user. The user is located at a preset position of the first video frame, the second video includes a plurality of second video frames, each of the second video frames includes the drone.
The shooting control method according to claim 4, wherein the target video comprises a plurality of first video frame groups and a plurality of second video frame groups, and the first video frame groups in the target video are the same as The second video frame group is adjacent, the first video frame group includes one or more first video frames, and the second video frame group includes one or more second video frames.
The shooting control method according to claim 1, wherein the method further comprises:

Controlling the terminal device to output shooting prompt information, and acquiring a shooting control instruction triggered by the user, wherein the shooting prompt information is used to prompt the user to shoot a video;

The acquiring the first video includes:

According to the shooting control instruction, controlling the first shooting device to shoot the user holding the terminal device to obtain a first video;

The obtaining of the second video includes:

According to the shooting control instruction, the second shooting device is controlled to shoot the drone to obtain a second video.
The shooting control method according to claim 6, wherein before the controlling the terminal device to output the shooting prompt information, the method further comprises:

controlling the terminal device to display a shooting control page, wherein the shooting control page includes a viewing angle switching icon and a first image collected by the first shooting device;

In response to the user's touch operation on the user in the first image, the user in the first image is used as the first photographing subject of the first photographing device;

In response to the user's triggering operation on the viewing angle switching icon, switching the first image in the shooting control page to the second image collected by the second shooting device;

In response to the user's triggering operation on the drone in the second image, the drone in the second image is used as the second photographing subject of the second photographing device;

When it is determined that the first photographing subject of the first photographing device is the user and the second photographing subject of the second photographing device is the drone, the terminal device is controlled to output photographing prompt information.
The shooting control method according to claim 7, wherein the method further comprises:

When the user does not exist in the first image, controlling the terminal device to output first prompt information, where the first prompt information is used to prompt the user to adjust the first shooting area of the first shooting device;

The posture of the first photographing device is adjusted according to a first posture adjustment instruction triggered by a user, and the first photographing area and the displayed first image change with the posture of the first photographing device.
The shooting control method according to claim 8, wherein the controlling the terminal device to output the first prompt information comprises:

acquiring first orientation information of the drone relative to the terminal device;

According to the first orientation information, the terminal device is controlled to output first prompt information.
The shooting control method according to claim 8, wherein the method further comprises:

When the user exists in the updated first image, second prompt information is output, wherein the second prompt information is used to prompt the user to stop adjusting the first shooting area.
The shooting control method according to claim 7, wherein the method further comprises:

When the drone does not exist in the second image, controlling the terminal device to output third prompt information, where the third prompt information is used to prompt the user to adjust the second shooting area of the second shooting device;

The posture of the second photographing device is adjusted according to the second posture adjustment instruction triggered by the user, and the second photographing area and the displayed second image change with the change of the posture of the second photographing device.
The shooting control method according to claim 11, wherein the controlling the terminal device to output the third prompt information comprises:

acquiring the second orientation information of the terminal device relative to the drone;

According to the second orientation information, the terminal device is controlled to output third prompt information.
The shooting control method according to claim 11, wherein the method further comprises:

When the drone exists in the updated second image, fourth prompt information is output, wherein the fourth prompt information is used to prompt the user to stop adjusting the second shooting area.
The shooting control method according to any one of claims 1-13, wherein the method further comprises:

in the process of controlling the first photographing device to photograph the user holding the terminal device, acquiring the flight trajectory in the photographing control instruction;

The drone is controlled to fly according to the flight trajectory, so that the drone is far away from the user holding the terminal device or close to the user holding the terminal device.
The shooting control method according to claim 14, wherein the flight trajectory includes any one of a first flight trajectory and a second flight trajectory, and the drone is controlled to fly according to the flight trajectory ,include:

controlling the drone to fly according to the first flight trajectory, so that the drone is far away from the user holding the terminal device;

or

The drone is controlled to fly according to the second flight trajectory, so that the drone is close to the user holding the terminal device.
The shooting control method according to any one of claims 1-13, wherein the method further comprises:

In the process of controlling the first photographing apparatus to photograph the user holding the terminal device, the drone is controlled to follow the user holding the terminal device to fly.
The photographing control method according to claim 16, wherein the controlling the drone to follow the user holding the terminal device to fly comprises:

obtaining the current image collected by the first photographing device;

The drone is controlled to follow the user holding the terminal device to fly according to the current image.
The photographing control method according to claim 17, wherein the controlling the drone to follow the user holding the terminal device to fly according to the current image comprises:

Inputting the current image into a preset target detection model to perform target detection on the user in the current image, and obtain target detection information of the user, wherein the target detection information includes the user's location in the world Size information, position information and angle information of the user relative to the drone in the coordinate system;

According to the target detection information and the preset target tracking algorithm, predict the target position information of the user at the next moment;

According to the target location information and the current location information of the drone, the drone is controlled to follow the user holding the terminal device to fly.
The photographing control method according to claim 16, wherein the controlling the drone to follow the user holding the terminal device to fly comprises:

obtaining the target rotation speed of the user holding the terminal device;

According to the target rotation speed, the drone is controlled to fly around and follow the user holding the terminal device.
The shooting control method according to claim 19, wherein the terminal device comprises an inertial measurement unit and a compass sensor, and the acquiring the target rotation speed of the user holding the terminal device comprises:

acquiring the first sensing data collected by the inertial measurement unit, and determining the target rotation speed of the user holding the terminal device according to the first sensing data;

or

acquiring the first sensing data collected by the inertial measurement unit and the second sensing data collected by the compass sensor;

According to the first sensing data and the second sensing data, a target rotation speed of the user holding the terminal device is determined.
The shooting control method according to claim 19, wherein the acquiring the target rotation speed of the user holding the terminal device comprises:

Acquiring images collected by the first photographing device at different moments, and determining the pixel coordinates of the identification target set on the terminal device in each of the images;

According to the pixel coordinates of the identification target in each of the images, the target rotation speed of the user holding the terminal device is determined.
The shooting control method according to claim 21, wherein the identification target comprises a reflection enhancement film provided on the terminal device.
The shooting control method according to claim 19, wherein the terminal device comprises an inertial measurement unit and a compass sensor, and the acquiring the target rotation speed of the user holding the terminal device comprises:

Obtain the first sensing data collected by the inertial measurement unit and the rotational speed of the identification target set on the terminal device;

determining the target rotation speed of the user holding the terminal device according to the first sensing data and the rotation speed of the identified target;

or

Acquiring the first sensing data collected by the inertial measurement unit, the second sensing data collected by the compass sensor, and the rotation speed of the marked target set on the terminal device;

According to the first sensing data, the second sensing data and the rotation speed of the identified target, the target rotation speed of the user holding the terminal device is determined.
The shooting control method according to claim 19, wherein the controlling the drone to fly around and following the user holding the terminal device according to the target rotation speed comprises:

obtain the surrounding flight angle of the UAV;

The drone is controlled to fly around and follow the user holding the terminal device according to the target rotation speed, until the flying angle of the drone around the user reaches the around flying angle.
The shooting control method according to claim 19, wherein the method further comprises:

In the process of controlling the drone to fly around and follow the user, if a throwing control command sent by the terminal device is detected, determining the throwing direction of the drone;

The drone is controlled to fly along the throwing direction.
The shooting control method according to claim 25, wherein the controlling the drone to fly along the throwing direction comprises:

Controlling the drone to fly along the throwing direction until the flying distance of the drone along the throwing direction reaches a preset distance;

or

Controlling the drone to fly along the throwing direction until the flying distance of the drone along the throwing direction reaches the throwing distance in the throwing control instruction;

or

The UAV is controlled to decelerate along the throwing direction until the horizontal flight speed of the UAV becomes zero.
The shooting control method according to claim 19, wherein the method further comprises:

During the process of controlling the drone to fly around and follow the user, the posture of the first photographing device is adjusted so that the user in the image captured by the first photographing device after the posture is adjusted is located in the Preset positions in the image.
A control device, characterized in that the control device is used to control an unmanned aerial vehicle and a terminal device, the unmanned aerial vehicle includes a first photographing device, the terminal device includes a second photographing device, and the control device includes a memory and processor;

the memory is used to store computer programs;

The processor is configured to execute the computer program and implement the following steps when executing the computer program:

acquiring a first video, where the first video is obtained by the control device controlling the first shooting device to shoot the user holding the terminal device according to a shooting control instruction triggered by the user;

acquiring a second video, where the second video is obtained by the control device controlling the second shooting device to shoot the drone according to the shooting control instruction;

The first video and the second video are synthesized to obtain a target video, where the target video includes at least a part of video frames in the first video and at least a part of video frames in the second video.
The control device according to claim 28, wherein the start and end shooting moments of the first video and the second video are the same.
The control device according to claim 28, characterized in that, during the process of photographing the user and the drone, the user is in the photographing area of the first photographing device, and the non- The man-machine is located in the shooting area of the second shooting device.
The control device according to claim 28, wherein the first video includes a plurality of first video frames, each of the first video frames includes the user, and all of the first video frames include the user. The user is located at a preset position of the first video frame, the second video includes a plurality of second video frames, each of the second video frames includes the drone.
The control device according to claim 31, wherein the target video includes a plurality of first video frame groups and a plurality of second video frame groups, and the first video frame group in the target video is the same as the The second video frame group is adjacent, the first video frame group includes one or more first video frames, and the second video frame group includes one or more second video frames.
The control device according to claim 28, wherein the processor is further configured to implement the following steps:

Controlling the terminal device to output shooting prompt information, and acquiring a shooting control instruction triggered by the user, wherein the shooting prompt information is used to prompt the user to shoot a video;

The acquiring the first video includes:

According to the shooting control instruction, controlling the first shooting device to shoot the user holding the terminal device to obtain a first video;

The obtaining of the second video includes:

According to the shooting control instruction, the second shooting device is controlled to shoot the drone to obtain a second video.
The control device according to claim 33, wherein before the processor controls the terminal device to output the shooting prompt information, it is further configured to:

controlling the terminal device to display a shooting control page, wherein the shooting control page includes a viewing angle switching icon and a first image collected by the first shooting device;

In response to the user's touch operation on the user in the first image, the user in the first image is used as the first photographing subject of the first photographing device;

In response to the user's triggering operation on the viewing angle switching icon, switching the first image in the shooting control page to the second image collected by the second shooting device;

In response to the user's triggering operation on the drone in the second image, the drone in the second image is used as the second photographing subject of the second photographing device;

When it is determined that the first photographing subject of the first photographing device is the user and the second photographing subject of the second photographing device is the drone, the terminal device is controlled to output photographing prompt information.
The control device according to claim 34, wherein the processor is further configured to implement the following steps:

When the user does not exist in the first image, controlling the terminal device to output first prompt information, where the first prompt information is used to prompt the user to adjust the first shooting area of the first shooting device;

The posture of the first photographing device is adjusted according to a first posture adjustment instruction triggered by the user, and the first photographing area and the displayed first image change with the posture of the first photographing device.
The control apparatus according to claim 35, wherein the controlling the terminal device to output the first prompt information comprises:

acquiring first orientation information of the drone relative to the terminal device;

According to the first orientation information, the terminal device is controlled to output first prompt information.
The control device according to claim 35, wherein the processor is further configured to implement the following steps:

When the user exists in the updated first image, second prompt information is output, wherein the second prompt information is used to prompt the user to stop adjusting the first shooting area.
The control device according to claim 34, wherein the processor is further configured to implement the following steps:

When the drone does not exist in the second image, controlling the terminal device to output third prompt information, where the third prompt information is used to prompt the user to adjust the second shooting area of the second shooting device;

The posture of the second photographing device is adjusted according to the second posture adjustment instruction triggered by the user, and the second photographing area and the displayed second image change with the change of the posture of the second photographing device.
The control apparatus according to claim 38, wherein the controlling the terminal device to output the third prompt information comprises:

acquiring the second orientation information of the terminal device relative to the drone;

According to the second orientation information, the terminal device is controlled to output third prompt information.
The control device according to claim 38, wherein the processor is further configured to implement the following steps:

When the drone exists in the updated second image, fourth prompt information is output, wherein the fourth prompt information is used to prompt the user to stop adjusting the second shooting area.
The control device according to any one of claims 28-40, wherein the processor is further configured to implement the following steps:

in the process of controlling the first photographing device to photograph the user holding the terminal device, acquiring the flight trajectory in the photographing control instruction;

The drone is controlled to fly according to the flight trajectory, so that the drone is far away from the user holding the terminal device or close to the user holding the terminal device.
The control device according to claim 41, wherein the flight trajectory includes any one of a first flight trajectory and a second flight trajectory, and the drone is controlled to fly according to the flight trajectory, include:

controlling the drone to fly according to the first flight trajectory, so that the drone is far away from the user holding the terminal device;

or

The drone is controlled to fly according to the second flight trajectory, so that the drone is close to the user holding the terminal device.
The control device according to any one of claims 28-40, wherein the processor is further configured to implement the following steps:

In the process of controlling the first photographing apparatus to photograph the user holding the terminal device, the drone is controlled to follow the user holding the terminal device to fly.
The control device according to claim 43, wherein the controlling the drone to follow the user holding the terminal device to fly comprises:

obtaining the current image collected by the first photographing device;

The drone is controlled to follow the user holding the terminal device to fly according to the current image.
The control device according to claim 44, wherein the controlling the drone to follow the user holding the terminal device to fly according to the current image comprises:

Inputting the current image into a preset target detection model to perform target detection on the user in the current image, and obtain target detection information of the user, wherein the target detection information includes the user's location in the world Size information, position information and angle information of the user relative to the drone in the coordinate system;

According to the size information, the angle information and the preset target tracking algorithm, predict the target position information of the user at the next moment;

According to the target location information and the current location information of the drone, the drone is controlled to follow the user holding the terminal device to fly.
The control device according to claim 43, wherein the controlling the drone to follow the user holding the terminal device to fly comprises:

obtaining the target rotation speed of the user holding the terminal device;

According to the target rotation speed, the drone is controlled to fly around and follow the user holding the terminal device.
The control device according to claim 46, wherein the terminal device comprises an inertial measurement unit and a compass sensor, and the acquiring the target rotation speed of the user holding the terminal device comprises:

acquiring the first sensing data collected by the inertial measurement unit, and determining the target rotation speed of the user holding the terminal device according to the first sensing data;

or

acquiring the first sensing data collected by the inertial measurement unit and the second sensing data collected by the compass sensor;

According to the first sensing data and the second sensing data, a target rotation speed of the user holding the terminal device is determined.
The control device according to claim 46, wherein the acquiring the target rotation speed of the user holding the terminal device comprises:

Acquiring images collected by the first photographing device at different times, and determining the pixel coordinates of the identification target set on the terminal device in each of the images;

According to the pixel coordinates of the identification target in each of the images, the target rotation speed of the user holding the terminal device is determined.
The control device according to claim 48, wherein the identification target comprises a reflection enhancement film provided on the terminal device.
The control device according to claim 46, wherein the terminal device comprises an inertial measurement unit and a compass sensor, and the acquiring the target rotation speed of the user holding the terminal device comprises:

Acquiring the first sensing data collected by the inertial measurement unit and the rotation speed of the identified target set on the terminal device;

determining the target rotation speed of the user holding the terminal device according to the first sensing data and the rotation speed of the identified target;

or

Acquiring the first sensing data collected by the inertial measurement unit, the second sensing data collected by the compass sensor, and the rotation speed of the marked target set on the terminal device;

According to the first sensing data, the second sensing data and the rotation speed of the identified target, the target rotation speed of the user holding the terminal device is determined.
The control device according to claim 46, wherein the controlling the drone to fly around and following the user holding the terminal device according to the target rotation speed comprises:

obtain the surrounding flight angle of the UAV;

The drone is controlled to fly around and follow the user holding the terminal device according to the target rotation speed, until the flying angle of the drone around the user reaches the surrounding flying angle.
The control device according to claim 46, wherein the processor is further configured to implement the following steps:

In the process of controlling the drone to fly around and follow the user, if a throwing control command sent by the terminal device is detected, determining the throwing direction of the drone;

The drone is controlled to fly along the throwing direction.
The control device according to claim 52, wherein the controlling the drone to fly along the throwing direction comprises:

Controlling the drone to fly along the throwing direction until the flying distance of the drone along the throwing direction reaches a preset distance;

or

Controlling the drone to fly along the throwing direction until the flying distance of the drone along the throwing direction reaches the throwing distance in the throwing control instruction;

or

Control the UAV to decelerate along the throwing direction until the horizontal flight speed of the UAV becomes zero.
The control device according to claim 46, wherein the processor is further configured to implement the following steps:

During the process of controlling the drone to fly around and follow the user, the posture of the first photographing device is adjusted so that the user in the image captured by the first photographing device after the posture is adjusted is located in the Preset positions in the image.
An unmanned aerial vehicle, characterized in that it includes:

body;

a power system, arranged on the body, for providing flight power for the drone;

a first photographing device, mounted on the body, for photographing a first video;

a first wireless communication device, provided on the body, for communicating with the second wireless communication device in the terminal device;

The control device according to any one of claims 28 to 54, configured to control the first photographing device to shoot a first video and/or the drone to fly.
A terminal device, characterized in that it includes:

a second shooting device for shooting a second video;

A display device for displaying images or a human-computer interface;

a second wireless communication device for communicating with the first wireless communication device in the drone;

The control device according to any one of claims 28-54, configured to control the second shooting device to shoot a second video.
A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the method described in any one of claims 1-27. The steps of the shooting control method described above.