WO2019062173A1

WO2019062173A1 - Video processing method and device, unmanned aerial vehicle and system

Info

Publication number: WO2019062173A1
Application number: PCT/CN2018/088723
Authority: WO
Inventors: 张伟
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2017-09-29
Filing date: 2018-05-28
Publication date: 2019-04-04

Abstract

Provided in the embodiments of the present invention are a video processing method and device, an unmanned aerial vehicle and a system, wherein the method comprises: controlling a photographing apparatus in an unmanned aerial vehicle to photograph an environment image at a first photographing point so as to obtain a first panoramic image, and obtaining a first video segment according to the first panoramic image; controlling the photographing apparatus in the unmanned aerial vehicle to photograph so as to obtain a second video segment during the movement of the unmanned aerial vehicle from the first photographing point to a second photographing point and from the second photographing point to a third photographing point according to a flight path; when the unmanned aerial vehicle reaches the third photographing point, controlling the photographing apparatus in the unmanned aerial vehicle to photograph an environment image so as to obtain a second panoramic image, and generating a third video segment according to the second panoramic image; and generating a target video according to the first video segment, the second video segment and the third video segment, which may achieve automatic and convenient photographing so as to obtain a video comprising at least two panoramic video segments.

Description

Video processing method, device, drone and system

Technical field

The present invention relates to the field of image processing technologies, and in particular, to a video processing method, device, drone, and system.

Background technique

With the development of multimedia technology, more and more users use video recording to record meaningful moments in life or at work, and share the captured videos with friends and family. Among them, the panoramic video has the advantage of having a large visual effect and providing users with more shooting environment information in the same video picture, which is favored by most users. However, in practical applications, professional photographers are required to use professional camera equipment to shoot, and the video produced by the camera is tediously post-production to obtain panoramic video. Therefore, how to effectively capture panoramic video has become a research hotspot.

Summary of the invention

The embodiment of the invention discloses a video processing method, device, drone and system, which can automatically and conveniently capture a video comprising at least two panoramic video segments.

In a first aspect, an embodiment of the present invention provides a video processing method, which is applied to a drone, where the drone is configured with a photographing device for capturing video, and the method includes: controlling shooting of the drone The device captures the environment image at the first shooting point to obtain a first panoramic image, and obtains a first video segment according to the first panoramic image, and moves the drone from the first shooting point to the second shooting point according to a flight trajectory. And during the moving from the second shooting point to the third shooting point, the shooting device controlling the drone captures a second video segment, and the drone reaches the third shooting point, and the control station The camera of the drone captures an environment image to obtain a second panoramic image, and generates a third video segment according to the second video segment, according to the first video segment, the second video segment, and the third video Segment, generate the target video.

In a second aspect, an embodiment of the present invention provides a video processing device, including a memory and a processor;

The memory is configured to store program instructions;

The processor executes the program instructions stored in the memory, and when the program instructions are executed, the processor is configured to perform the following steps:

The photographing device that controls the drone captures an environment image at a first photographing point to obtain a first panoramic image, and obtains a first video segment according to the first panoramic image;

Controlling the shooting of the drone by the drone during the movement of the drone from the first photographing point to the second photographing point and from the second photographing point to the third photographing point Two video segments;

When the drone reaches the third shooting point, the camera of the drone is controlled to capture an environment image to obtain a second panoramic image, and a third video segment is generated according to the second panoramic image;

Generating a target video according to the first video segment, the second video segment, and the third video segment.

In a third aspect, an embodiment of the present invention provides a drone, including:

body;

a power system disposed on the fuselage for providing flight power;

a processor, configured to control the camera of the drone to capture an environment image at a first shooting point to obtain a first panoramic image, and obtain a first video segment according to the first panoramic image, and follow the flight path in the drone Controlling the camera of the drone to capture a second video segment during the movement from the first photographing point to the second photographing point and from the second photographing point to the third photographing point The drone reaches the third shooting point, controls the shooting device of the drone to capture the environment image to obtain a second panoramic image, and generates a third video segment according to the second panoramic image, according to the first video segment. And generating, by the second video segment and the third video segment, a target video.

In a fourth aspect, an embodiment of the present invention provides a video processing system, where the system includes: a video processing device and a drone;

The video processing device is configured to send a flight control instruction to the aircraft, where the flight control instruction is used to instruct the drone to fly according to the determined flight trajectory;

The drone is configured to control the drone to fly according to the flight trajectory and control a photographing device mounted on the drone to perform shooting according to the flight control instruction;

The video processing device is further configured to control the camera of the drone to capture an environment image at a first shooting point to obtain a first panoramic image, and obtain a first video segment according to the first panoramic image, Controlling the camera of the drone to capture a second video segment during movement of the flight path from the first shooting point to the second shooting point and from the second shooting point to the third shooting point When the drone arrives at the third shooting point, the camera of the drone is controlled to capture an environment image to obtain a second panoramic image, and a third video segment is generated according to the second panoramic image, according to the The first video segment, the second video segment, and the third video segment generate a target video.

In a fifth aspect, an embodiment of the present invention provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the computer program is executed by the processor, implements the video processing method according to the first aspect.

The embodiment of the present invention can perform shooting control around the drone during the movement of the drone according to the flight trajectory, and obtain a first video segment based on the panoramic image, a second video segment, and a third video segment based on the panoramic image, and Automatically and conveniently generate a target video, which can generate a video including at least two panoramic video segments for the user in combination with the flight trajectory, and can realize a smooth transition between the panoramic video segment and the non-panoramic video segment, and realize multiple panoramic video segments. Smooth transition between users to meet the user's need for automation, intelligent shooting and processing of video.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings to be used in the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without paying for creative labor.

1 is a schematic diagram of a flight trajectory provided by an embodiment of the present invention;

2 is a schematic diagram of another flight trajectory provided by an embodiment of the present invention;

3 is a schematic flowchart of an image mosaic method according to an embodiment of the present invention;

4 is a schematic diagram of a video generation process according to an embodiment of the present invention;

FIG. 5 is a schematic flowchart of generating a panoramic image according to an embodiment of the present invention;

6 is a schematic structural diagram of a UAV system according to an embodiment of the present invention;

FIG. 7 is a schematic flowchart diagram of a video processing method according to an embodiment of the present disclosure;

FIG. 8 is a schematic flowchart diagram of another video processing method according to an embodiment of the present disclosure;

FIG. 9 is a schematic structural diagram of a video processing device according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

Some embodiments of the present invention are described in detail below with reference to the accompanying drawings. The features of the embodiments and examples described below can be combined with each other without conflict.

The video processing method provided by the embodiment of the present invention may be performed by a video processing device, and the video processing device may be disposed on a drone capable of capturing video, or may be disposed on a remote controller at the ground end. The video processing method can be applied to a video shooting task based on a drone. In other embodiments, a video shooting task of a movable device such as a robot capable of autonomous movement can also be applied, and the following video corresponding to the drone is used. The processing method is illustrated by way of example.

The video processing method proposed by the scheme adopts a video processing scheme combined with the trajectory, which is applied to the drone, and can be mounted by being in the process of autonomous flight according to the determined flight trajectory by determining the flight trajectory. The camera on the drone captures images at various waypoint locations. Specifically, when the first position of the flight path is reached, the drone can be controlled to rotate at the position or control the pan/tilt of the drone to perform circumferential rotation, and the environment image is captured during the rotation, and each of the captured images will be captured. The environment image is spliced to obtain a first panoramic image, and the first video segment is generated according to the first panoramic image; the second video segment composed of the captured environmental image may be acquired at the second position on the flight trajectory; When the third position of the flight path is selected, the drone can be controlled to rotate at the position or control the head of the drone to perform circumferential rotation, and the environment image is captured during the rotation, and the captured environmental images are stitched. Obtaining a second panoramic image, generating a third video segment according to the second panoramic image; generating a target video according to the first video segment, the second video segment, and the third video segment, where the target video includes not only at least two synthesized by the panoramic image The panoramic video segments (ie, the first video segment and the third video segment) further include a non-panoramic video segment (ie, a second video segment), and the panoramic video segment has a comparison The large viewing angle provides the user with a global visual effect of the shooting area, and the non-panoramic image can provide the user with a partial visual effect of the shooting area. Therefore, in the embodiment of the present invention, a video including at least two panoramic video segments can be generated for the user by combining the flight trajectory, and the user can provide the global visual effect and the partial visual effect of the shooting region, and can realize the panoramic video segment and the non-panoramic video segment. Smooth transition between the two, and achieve a smooth transition between multiple panoramic video segments to meet the user's need for automation, intelligent shooting and processing of video.

It should be noted that the flight trajectory of the embodiment of the present invention refers to a flight trajectory of the camera of the drone during the process of capturing an image, and the flight trajectory may be a non-closed flight trajectory or a closed flight trajectory. A closed flight path refers to a flight path where the starting point of the flight and the end point of the flight are not at the same point in the process of capturing the image. The closed flight path means that the starting point of the flight is the same as the ending point of the flight in the process of capturing the image. The flight path of the location point.

In one embodiment, the flight trajectory may be a non-closed flight trajectory in which the drone flies and takes an image. As shown in FIG. 1, FIG. 1 is a schematic diagram of a flight trajectory according to an embodiment of the present invention. 11 denotes a drone, 12 denotes a position point A, 13 denotes a position point B, 14 denotes a position point B, 15 denotes a user, and the user 15 can determine the position point A and the position point B, and the position point C by the video processing device The flight path is determined according to the position point A, the position point B and the position point C. The video processing device can control the drone to fly along the flight path composed of the position point A, the position point B and the position point C, and hang the hook. The photographing device mounted on the drone 11 captures an image during flight. Specifically, after determining the flight path composed of the position point A, the position point B, and the position point C, the video processing device can control the drone 11 to fly according to the flight path from any position in the flight path, and control the drone. 11 rotating at the first position or controlling the pan/tilt head of the drone 11 to perform circumferential rotation, and taking an environment image during the rotation process, splicing the captured environmental images to obtain a first panoramic image, according to the first The panoramic image generates a first video segment; a second video segment on the flight path can acquire a second video segment composed of the captured environmental image; and when the third location point of the flight trajectory is reached, the drone 11 can be controlled The position point rotates or controls the pan/tilt head of the drone 11 to perform circumferential rotation, and captures an environment image during the rotation process, and splices the captured environment images to obtain a second panoramic image, which is generated according to the second panoramic image. a third video segment; generating a target video according to the first video segment, the second video segment, and the third video segment.

For example, the drone 11 flies out from the position point A along the flight path shown in FIG. 1, and flies past the position point B along the position point A and reaches the position point C. If the drone reaches the position point A, the control is performed. The drone 11 rotates at the position point A or controls the pan/tilt head of the drone 11 to perform circumferential rotation, and captures an environment image during the rotation process, and splices the captured environment images to obtain a first panoramic image, according to which The first panoramic image generates a first video segment; when the drone moves from the position point A to the position point B according to the flight trajectory, and from the position point B to the position point C, the camera of the drone 11 can be controlled to perform Shooting to obtain the second video segment; when the drone reaches the position point C, controlling the drone 11 to rotate at the position point C or controlling the pan-tilt of the drone 11 to perform circumferential rotation, and taking an environment image during the rotation, The captured respective environment images are spliced to obtain a second panoramic image, and the third video segment is generated according to the second panoramic image; and the target video is generated according to the first video segment, the second video segment, and the third video segment.

For another example, the drone 11 flies out from the position point B along the flight path shown in FIG. 1, and flies along the position point B to the position point A, and controls the drone 11 to rotate at the position point A or control the unmanned person. The pan/tilt of the machine 11 is rotated circumferentially, and an environment image is taken during the rotation process, and the captured environmental images are spliced to obtain a first panoramic image, and the first video segment is generated according to the first panoramic image; The machine moves according to the flight path from the position point A to the position point B, and from the position point B to the position point C, the shooting device of the drone 11 can be controlled to obtain the second video segment; the drone arrives at the position At point C, the drone 11 is controlled to rotate at the position point C or to control the pan/tilt head of the drone 11 to perform circumferential rotation, and to capture an environment image during the rotation process, and to splicing the captured environmental images to obtain a second panorama. And generating, according to the second panoramic image, a third video segment; generating a target video according to the first video segment, the second video segment, and the third video segment.

The embodiment for acquiring the second video segment further includes: in the process of moving the drone from the position point A to the position point B according to the flight trajectory, the camera unit of the drone 11 can be controlled to capture the sub-video segment 1 in the absence of During the movement of the man-machine from the position point B to the position point C according to the flight path, the sub-video segment 1 and the sub-video segment 2 can be spliced to obtain the second video segment. .

In another embodiment, the flight trajectory may be a closed flight trajectory in which the drone flies and takes an image. As shown in FIG. 2, 11 denotes a drone, 16 denotes a position point D, 17 denotes a position point E, 18 denotes a position point F, 19 denotes a position point G, 15 denotes a user, and the user 15 can determine by a video processing device The position point D and the position point E, the position point F and the position point G determine the flight trajectory according to the position point D and the position point E, the position point F and the position point G, and the video processing device can control the drone along the position A flight path composed of a point D, a position point E, a position point F, and a position point G is flew, and an image during flight is captured by a photographing device mounted on the drone 11. Specifically, after determining the flight path composed of the position point D, the position point E, the position point F, and the position point G, the video processing device may control the drone 11 to fly according to the flight path from any position in the flight path. Controlling the first position of the drone 11 to rotate or controlling the pan/tilt head of the drone 11 to perform circumferential rotation, and capturing an environment image during the rotation process, and splicing the captured environment images to obtain a first panoramic image, according to which The first panoramic image generates a first video segment; the second location point on the flight trajectory may acquire a second video segment composed of the captured environmental image; and when the third location point of the flight trajectory is reached, the drone may be controlled 11 rotate at the position or control the pan/tilt head of the drone 11 to perform circumferential rotation, and take an environment image during the rotation process, and splicing the captured environmental images to obtain a second panoramic image according to the second panoramic view. The image generates a third video segment; the fourth position on the flight trajectory may acquire a fourth video segment composed of the captured environmental image; Frequency band, a second video segment, the third and fourth video segment to generate a target video video segment.

For example, the drone 11 flies out from the position point D along the flight trajectory shown in FIG. 2, flies through the position point E along the position point D and reaches the position point F, and then flies past the position point F along the position point F. G and return to the position point D, if the drone reaches the position point D, control the drone 11 to rotate at the position point D or control the pan-tilt of the drone 11 to make a circumferential rotation, and take an environmental image during the rotation, The captured respective environmental images are spliced to obtain a first panoramic image, according to which the first panoramic image is generated, and the unmanned aerial vehicle moves from the position point D to the position point E according to the flight trajectory, and arrives from the position point E. During the process of the position point F, the photographing device of the drone 11 can be controlled to obtain a second video segment; when the drone reaches the position point F, the drone 11 is controlled to rotate at the position point F or the drone 11 is controlled. The pan/tilt is rotated circumferentially, and the environment image is taken during the rotation process, and the captured environmental images are spliced to obtain a second panoramic image, and the third video segment is generated according to the second panoramic image; Flight track During the movement of the track from the position point F to the position point G and returning from the position point G to the position point D, the photographing device of the drone 11 can be controlled to take a fourth video segment, and the drone returns to the position point D. The target video is generated according to the first video segment, the second video segment, the third video segment, and the fourth video segment.

It should be noted that the video processing device in the embodiment of the present invention may use a trajectory generation control algorithm to determine a flight trajectory of the drone. The trajectory generation control algorithm mainly plans to capture a second video segment from the flight start point to the end point. The flight path, and control the drone to rotate at two locations, and control the camera to capture the environment image during the rotation to obtain a panoramic image. The trajectory generation control algorithm of the embodiment of the present invention may include at least one of a Bezier curve or a 5-order-B-spline, etc., which is not limited by the embodiment of the present invention.

In the embodiment of the present invention, the video processing device can determine a flight path of any shape linked by three position points, such as a straight line, a curve, a spiral line, etc., by setting three position points. In the process of determining the flight path of the drone, FIG. 1 can be used as an example. As shown in FIG. 1, the curve trajectory from the position point A through the position point B to the position point C is a determined flight trajectory.

It should be noted that since the shooting device of the drone is not an ultra wide-angle shooting device but an ordinary shooting device, the viewing angle of the shooting device is limited, and only a small angle of view image can be taken at a time, so it is necessary to plan the shooting angle and cover more. A wide range of perspectives. For example, suppose that in the form of a circumferential scan, the pitch angle of the UAV pan/tilt can be fixed first, and then the yaw angle of the yaw angle is controlled to control the environment image of the shooting device for one week, and then the pitch angle is gradually changed. The angle yaw angle controls the camera to capture different environmental images.

In the embodiment of the present invention, the video processing device may obtain a panoramic image by splicing the captured environmental images. Specifically, the video processing device may use a splicing algorithm to implement image splicing. As shown in FIG. 3, the image splicing is performed. The process includes:

S21. Find feature matching.

S22, BA optimization.

S23, compensation exposure.

S24. Find the stitching line.

In steps S21-S24, when the splicing starts, first, the head of the drone can be rotated at different pitch angles, and multiple images can be captured by the photographing device, and the images have a certain overlap ratio, such as The overlap ratio is generally between 0.2 and 0.3. In order to avoid overlapping parts in the panoramic image after splicing, the features of each two adjacent images can be searched for and matched, and the matched features are subjected to beam adjustment (Bundle Adjustment, BA). Optimization, making the relative position between the images more accurate, then performing exposure compensation for each image, and finding the stitching line of each two adjacent images, and finally transforming all the images into a panoramic image by curve warp deformation, this The two adjacent images at the location may refer to the two images with the shortest shooting time interval, and the panoramic image herein may refer to the first panoramic image or the second panoramic image.

It should be noted that, in the embodiment of the present invention, the imaging device is calibrated, and the calibration value is used as the initial value of the parameters in the imaging device, and the targeted feature points can be matched according to the captured environmental images, thereby reducing The error generated during the matching process.

In the embodiment of the present invention, after obtaining the target panoramic image (the first panoramic image or the second panoramic image), the video processing device may generate the target video segment (the first video segment or the third video segment) according to the target panoramic image, and FIG. 4 For example, FIG. 4 is a schematic diagram of a video generation process according to an embodiment of the present invention, where 31 represents a first perspective point of a target panoramic image, 32 represents a first imaging point, and 33 represents a first panoramic point, 37 Represents a first projection plane, 34 represents a second perspective point, 35 represents a second panoramic point, 36 represents a second imaging point, and 38 represents a second projection plane. The panoramic projection similar to the asteroid is actually the first viewing point shown in FIG. 4, and the projection plane is at the first projection plane 37, that is, a projection mode similar to the center of the asteroid, and normal. The photograph of the viewing angle is a projection of the viewing point at the second viewing point 34, ie at the center of the circle resembling an asteroid, the projection plane at the second projection plane 38, by moving the viewpoint point from the first viewing point 31 to The second viewing point 34, while moving the projection plane from the first projection plane 37 to the second projection plane 38, can gradually expand the target panoramic image to obtain the target video segment.

Referring to FIG. 5, FIG. 5 is a schematic flowchart of generating a panoramic image according to an embodiment of the present invention. As shown in FIG. 5, the video processing device first determines a flight trajectory of the drone, and then controls the drone to fly along the determined flight trajectory, and controls the camera of the drone at a certain point (eg, When a shooting point or a third shooting point is taken, an image of each orientation is taken, and the images in the respective orientations are stitched to obtain a spherical panoramic image (such as obtaining a first panoramic image or a second panoramic image).

It should be noted that the first location point, the second location point, the third location point, and the fourth location point are all location points on the flight path of the drone, and the location points may be different, and the video processing device may be based on the user. The selection operation or setting of each shooting point according to the user's needs, the video processing device can also automatically set each location point according to flight parameters (such as flight altitude, flight distance or flight time, etc.).

Referring to FIG. 6, FIG. 6 is a schematic structural diagram of a UAV system according to an embodiment of the present invention. The system includes a video processing device 51 and a drone 52. The video processing device 51 may be a control terminal of the drone, and may specifically be one of a remote controller, a smart phone, a tablet computer, a laptop computer, a ground station, a wearable device (watch, a wristband) or In many cases, the drone 52 may be a rotary wing type unmanned aerial vehicle, such as a quadrotor drone, a six-rotor drone, an eight-rotor drone, or a fixed-wing drone. The drone includes a power system 521 for providing flight power to the drone, wherein the power system 521 includes one or more of a propeller, a motor, and an electric power, and the drone may further include a pan/tilt 522 and The imaging device 523 and the imaging device 523 are mounted on the main body of the drone through the pan/tilt 522. The camera is used for image or video shooting during the flight of the drone, including but not limited to multi-spectral imager, hyperspectral imager, visible light camera and infrared camera. The pan/tilt is multi-axis transmission and stabilization system. The pan/tilt motor compensates the shooting angle of the imaging device by adjusting the rotation angle of the rotating shaft, and prevents or reduces the jitter of the imaging device by setting an appropriate buffer mechanism.

In one embodiment, the video processing device 51 can configure an interaction device that interacts with the user, and the interaction device can be one or more of a touch display screen, a keyboard, a button, a joystick, and a pulsator. A user interface may be provided, and the user may perform a click operation on the user interface of the video processing device 51, the click operation may confirm a location point by clicking once, and the video processing device 51 receives the three locations clicked by the click operation. After the point, the video processing device 51 connects the three location points to form a flight trajectory connected by the three points, and the flight trajectory may be a linear trajectory or a curved trajectory, thereby controlling the drone 52 to fly according to the flight trajectory. In order to cause the photographing device 523 to move in accordance with the flight trajectory. It should be noted that the click operation on the user interface is only one method for confirming the flight trajectory, and the manner for confirming the flight trajectory is not limited in the embodiment of the present invention.

In an embodiment, during the movement of the drone according to the flight path, the video processing device 51 may control the camera of the drone to capture the environment image at the first shooting point to obtain the first panoramic image, and according to the first The panoramic image generates a first video segment; during the movement of the drone from the first shooting point to the second shooting point and from the second shooting point to the third shooting point according to the flight trajectory, controlling the shooting device of the drone Obtaining a second video segment; when the drone reaches the third shooting point, controlling the camera of the drone to capture an environment image to obtain a second panoramic image, and generating a third video segment according to the second panoramic image; A target video is generated according to the first video segment, the second video segment, and the third video segment.

Referring to FIG. 7, FIG. 7 is a schematic flowchart diagram of a video processing method according to an embodiment of the present invention. The method may be performed by a video processing device, where a specific explanation of the video processing device is as described above. Specifically, the method of the embodiment of the present invention includes the following steps.

S701. The photographing device that controls the drone captures the environment image at the first photographing point to obtain a first panoramic image, and obtains the first video segment according to the first panoramic image.

In the embodiment of the present invention, in order to provide the user with the global visual effect of the shooting area and provide the user with more information about the shooting area, the video processing device can control the shooting device of the drone to capture the environmental image at the first shooting point. a first panoramic image, and obtaining a first video segment based on the first panoramic image. Specifically, the video processing device may first determine three location points, determine a flight path of the drone according to the three location points, and, after determining the flight path of the drone, cause the drone to perform motion according to the determined flight path. During the motion, the video processing device can control the camera of the drone to capture the environment image to obtain the first panoramic image when the first shooting point is reached, and obtain the first video segment according to the first panoramic image. Taking FIG. 1 as an example, for example, the video processing device first determines three position points A, B, and C, and determines a connection curve between the position point A, the position point B, and the position point C as a flight path of the drone. The video processing device controls the drone to fly according to the flight path composed of the position point A, the position point B and the position point C shown in FIG. 1, and the drone can fly along the position point A through the position point B and reach the position point C. During the flight according to the flight path, when the drone reaches the position point A (the first shooting point), the video processing device can control the drone or control the head of the drone to perform the circle at the position point A. Rotating, the photographing device controlling the drone photographs the environment image during the rotation of the drone to obtain a first panoramic image, thereby generating a first video segment according to the first panoramic image.

In an embodiment, the video processing device may control the drone to rotate at a first shooting point to capture a first panoramic image, and specifically, control the drone to make a clockwise direction at the first shooting point. Rotating circumferentially, or controlling the drone to make a circumferential rotation in a counterclockwise direction to obtain a first panoramic image. For example, as shown in FIG. 1, when the drone reaches the position point A, the video processing device can control the drone to perform a circumferential rotation in the clockwise direction at the position point A, or the video processing device can control the drone at The position point A is circumferentially rotated in the counterclockwise direction, and the photographing device is controlled to perform the shooting to obtain the first panoramic image during the circumferential rotation.

In an embodiment, after acquiring the first panoramic image, the video processing device may perform an expansion process on the first panoramic image, and generate a first video segment according to each image obtained in the expansion process. The process of generating the first video segment according to the first panoramic image may be illustrated by using FIG. 4 as an example. Specifically, the acquired first panoramic image is used as the spherical panoramic image in FIG. 4, first from the first panoramic image. The first view point 31 (the north pole point) starts to be gradually expanded, and the unfolding manner is to gradually move the view point from the first view point 31 to the second view point 34 (the center of the sphere) of the spherical panoramic image, and expand the first panorama. The image, at the same time, moves the projection plane following the point of view point from the spherical center plane 37 (first projection plane) of the spherical panoramic image to the plane 38 (second projection plane) where the south pole of the spherical panoramic image lies, thereby The first panoramic image is progressively expanded to generate a first video segment.

S702. During the movement of the drone from the first photographing point to the second photographing point and from the second photographing point to the third photographing point according to the flight path, the photographing device controlling the drone is photographed to obtain the second Video segment.

In the embodiment of the present invention, in the process of moving the drone from the first photographing point to the second photographing point and from the second photographing point to the third photographing point according to the flight path, in order to provide the user with a part of the photographing area A visual effect to show a local feature of the shooting area (such as an iconic building or a character), the video processing device can control the camera of the drone to capture a second video segment. Specifically, as shown in FIG. 1 , it is assumed that the flight path of the drone is as shown in FIG. 1 , and the flight path is a curved track between the position point A, the position point B, and the position point C, in the drone. During the movement according to the flight trajectory, when the drone flies from the position point A through the position point B (second shooting point) and reaches the position point C (third shooting point), the drone is controlled. The camera captures a second video segment.

In an embodiment, the first shooting point, the second shooting point, and the third shooting point may be three position points on the flight path. For example, as shown in FIG. 1 , the first shooting point may be It is the position point A on the flight path, the second shooting point may be the position point B on the flight path, and the third shooting point may be the position point C on the flight path.

In one embodiment, the first shooting point may be a first position point selected according to the received selection operation on the flight path, and the second shooting point may be based on the received flight path. The second position point selected by the operation is selected, and the third shooting point is a third position point selected according to the received selection operation on the flight path.

In one embodiment, the first photographing point is a position point of the drone when the drone tracks the target object reaching the first position point; the second photographing point is that the drone tracks the target object The position point of the drone when the second position point is reached; the third shooting point is the position point of the drone when the drone tracks the target object to reach the third position point, where the target object is located A moving object or a moving person or animal.

It should be noted that the video processing device can plan the trajectory of any shape by the setting of the shooting point, such as a straight line, a curve, a spiral, etc., and then discretize the entire curve as a passing point of the drone, guiding the unmanned person. Flight of the aircraft.

S703. After the drone reaches the third shooting point, control the shooting device of the drone to capture an environment image to obtain a second panoramic image, and generate a third video segment according to the second panoramic image.

In the embodiment of the present invention, in order to provide the user with the global visual effect of the shooting area and provide the user with more information about the shooting area, the video processing device can control the shooting device of the drone to capture the environmental image at the third shooting point. a second panoramic image, and obtaining a third video segment based on the second panoramic image.

In an embodiment, the video processing device may control the drone to rotate at a third shooting point to capture a second panoramic image. Specifically, the drone may be controlled to be clockwise at the third shooting point. Rotating circumferentially, or controlling the drone to make a circumferential rotation in a counterclockwise direction to obtain a second panoramic image. For example, as shown in FIG. 1, when the drone reaches the position point C, the video processing device can control the drone to perform a circumferential rotation in the clockwise direction at the position point C, or the video processing device can control the drone at The position point C is circumferentially rotated in the counterclockwise direction, and the photographing device is controlled to perform the second panoramic image during the circumferential rotation.

In an embodiment, after acquiring the second panoramic image, the video processing device may perform an expansion process on the second panoramic image, and generate a third video segment according to each image obtained in the expansion process. The process of generating the third video segment according to the second panoramic image may be illustrated by using FIG. 4 as an example. Specifically, the acquired second panoramic image is used as the spherical panoramic image in FIG. 4, first from the second panoramic image. The first view point 31 (the north pole point) starts to be gradually expanded, and the unfolding manner is to gradually move the view point from the first view point 31 to the second view point 34 (the center of the sphere) of the spherical panoramic image, and expand the second panorama. The image, at the same time, moves the projection plane following the point of view point from the spherical center plane 37 (first projection plane) of the spherical panoramic image to the plane 38 (second projection plane) where the south pole of the spherical panoramic image lies, thereby The second panoramic image is gradually expanded to generate a third video segment.

In one embodiment, in order to achieve a smooth transition between the first video segment and the second video segment, and a smooth transition between the second video segment and the third video segment, thereby implementing a panoramic video segment (the first video segment or the third video) a smooth transition between the segment) and the non-panoramic video segment (the second video segment), and a smooth transition between the plurality of panoramic video segments (eg, the first video segment and the second video segment), the video processing device may The last image of a panoramic image serves as the first video frame of the second video segment, and the last video frame of the second video segment serves as the first image of the second panoramic image.

S704. Generate a target video according to the first video segment, the second video segment, and the third video segment.

In the embodiment of the present invention, the video processing device may generate a target video according to the first video segment, the second video segment, and the third video segment.

In the embodiment of the present invention, the video processing device may control the camera of the drone to capture the environment image at the first shooting point to obtain the first panoramic image, and generate the first video segment according to the first panoramic image; During the movement of the flight path from the first shooting point to the second shooting point and from the second shooting point to the third shooting point, the camera controlling the drone captures the second video segment; the drone arrives at the a third shooting point, controlling a shooting device of the drone to capture an environment image to obtain a second panoramic image, and generating a third video segment according to the second panoramic image; according to the first video segment, the second video segment, and the third The video segment generates a target video, and can generate a video including at least two panoramic video segments for the user in combination with the flight trajectory, and can realize a smooth transition between the panoramic video segment and the non-panoramic video segment, and realize a plurality of panoramic video segments. Smooth transition, can provide users with a global visual effect and local visual effects of a certain area, to meet the user's automation, intelligent shooting and video Demand.

Referring to FIG. 8, FIG. 8 is a schematic flowchart diagram of another video processing method according to an embodiment of the present invention. The method may be performed by a video processing device, where a specific explanation of the video processing device is as described above. The method of the embodiment of the present invention is different from the embodiment described above in FIG. 7 in that the present embodiment determines the flight path of the drone by acquiring the first location point, the second location point, and the third location point, and controls the cloud. The table or the drone rotates to control the photographing device to take an environment image during the rotation of the drone or the pan/tilt, and splicing the respective environment images to obtain a panoramic image.

S801. Acquire a first location point, a second location point, and a third location point.

In the embodiment of the present invention, the first location point, the second location point, and the third location point may be determined by the video processing device according to the three sets of latitude and longitude information input by the user, or may be determined according to the location information input by the user. The location information input by the user here may refer to the name of a certain building (XX building) or a certain place name (XX of the XX city), or may be determined by receiving a click operation of the user, or may be based on a drone flight. The distance set may also be set according to the altitude of the drone flight. For example, if the flying height reaches the first preset height threshold (for example, 50 m) as the first position point, the flying height reaches the second preset. A position point of a height threshold (for example, 30 m) is used as a second position point, and a position point at which the flying height reaches a third preset height threshold (for example, 60 m) is used as a third position point, where the first preset height threshold, the third pre-predetermined The height threshold may be greater than the second preset height threshold. The first preset height threshold and the third height threshold may be equal or not equal to each other, which is not limited by the present invention.

Specifically, FIG. 1 is used as an example. When the video processing device receives the click operation of the user, the location point A as shown in FIG. 1 may be obtained as the first location point, and the location point B is the second location point and the location point. C is the third position point. It should be noted that the first location point, the second location point, and the third location point may be any point of the user clicking operation, for example, as shown in FIG. 1, the first location point, the second location point, and the The three-position point may be the position point A, the position point B, and the position point C, or may be a point at any other position. The determination of the position point on the flight path is not limited in the embodiment of the present invention.

S802. Determine, according to the location information of the first location point and the second location point, a first flight trajectory of the drone.

In an embodiment, the video processing device may generate, according to the determined location information of the first location point and the second location point, the first trajectory and the second photographic point according to the first trajectory generation rule. A first straight track between and determines the first straight track as a first flight path. Here, the first trajectory generation rule may include a trajectory generation control algorithm, and the determination of the flight trajectory of the drone usually has a plurality of commonly used algorithms, such as a Bezier curve, a five-time B-spline curve, and a 5-order-B-spline algorithm. .

In one embodiment, according to the location information of the first location point and the second location point, generating a first curved trajectory from the first shooting point and the second shooting point according to a second trajectory generation rule And determining the first curve trajectory as the first flight trajectory. Here, the second trajectory generation rule may include a trajectory generation control algorithm, and the determination of the flight trajectory of the drone usually has a plurality of commonly used algorithms, such as a Bezier curve, a 5-time B-spline curve, and a 5-order-B-spline algorithm. . Specifically, FIG. 1 is taken as an example, and a curved trajectory between the position point A and the position point B may be used as the first flight trajectory.

S803. Determine, according to the location information of the second location point and the third location point, a second flight trajectory of the drone.

In one embodiment, according to the position information of the second position point and the third position point, generating a second straight line trajectory from the second shooting point and the third shooting point according to the first trajectory generation rule; The second linear trajectory is determined to be a second flight trajectory.

In one embodiment, according to the second position point and the position information of the third position point, according to the second trajectory generation rule, generating a second curved trajectory from the third shooting point and the fourth shooting point, The second curved trajectory is determined as the second flight trajectory. Specifically, FIG. 1 is taken as an example, and a curved trajectory between the position point B and the position point C may be used as the second flight trajectory.

In one embodiment, the first shooting point and the second shooting point are two time points in the process of the drone moving in accordance with the first flight path, and the second shooting point and the third shooting point are drones. At two points in time during the movement in accordance with the second flight path. Specifically, the video processing device may determine three time points of the shooting time interval by setting a shooting time interval, and use the determined three time points as the first shooting point, the second shooting point, and the third shooting point. For example, the video processing device may preset the shooting time interval of the camera of the drone to be 5 seconds, and the first shooting point is the time point 10:00:05, so that the time point of the second shooting point can be determined to be 10:00: 10. The time point for determining the third shooting point is 10:00:15.

S804. The first position point is used as the first shooting point, and the second position point is used as the second shooting point, and the third position point is used as the third shooting point.

In the embodiment of the present invention, the video processing device may use the first location point as the first shooting point, the second location point as the second shooting point, and the third location point as the third shooting point. The first shooting point, the second shooting point, and the third shooting point may be three position points on the flight path.

S805. The photographing device that controls the drone captures the environment image at the first photographing point to obtain a first panoramic image, and obtains the first video segment according to the first panoramic image.

In one embodiment, the video processing device may control the pan-tilt rotation, wherein the camera of the drone follows the pan-tilt rotation to obtain the first panoramic image. Specifically, the video processing device may adjust a pitch angle of the unmanned aerial platform to a first pitch angle; control the pan-tilt to perform circumferential rotation by rotating a yaw angle of the pan-tilt, and acquire the camera And capturing a first environment image during the following rotation of the motion; adjusting a pitch angle of the pan/tilt to a second pitch angle after controlling the rotation of the motion; and rotating the yaw angle of the pan/tilt Controlling the pan-tilt to perform circumferential rotation, and acquiring a second environment image captured by the photographing device during the following rotation of the motion; splicing the first environment image and the second environment image And obtaining the first panoramic image.

In one embodiment, acquiring a first feature point set of the first environment image and a second feature point set of the second environment image, according to the first feature point set and the second feature point set Feature point matching to obtain a first target feature point set, and determining, according to the first target feature point set, a first stitching line of the first environment image and the second environment image, according to the first stitching line, The first environment image and the second environment image are spliced to obtain the first panoramic image.

It should be noted that, in the embodiment of the present invention, when the first environment image and the second environment image are spliced, the camera of the drone is calibrated, and the calibration value is used as the initial value of the parameter in the camera. Using the inertial measurement unit imu information as the initial value of the ba algorithm, the number of iterations can be reduced, and the relationship between the two characteristic images of the respective images can be used to perform matching of the targeted feature points, thereby reducing the matching error.

In one embodiment, the video processing device can control the rotation of the drone; wherein the camera of the drone follows the rotation of the drone to obtain the first panoramic image. Specifically, in the process of controlling the rotation of the drone, the video processing device may control the UAV to perform circumferential rotation in a clockwise direction, or control the UAV to perform circumferential rotation in a counterclockwise direction, The camera of the drone follows the rotation of the drone to obtain a first panoramic image.

S806. During the movement of the drone from the first photographing point to the second photographing point and from the second photographing point to the third photographing point according to the flight path, the photographing device controlling the drone is photographed to obtain the second Video segment.

S807. When the drone reaches the third shooting point, the camera of the drone is controlled to capture an environment image to obtain a second panoramic image, and a third video segment is generated according to the second panoramic image.

In one embodiment, the video processing device can control pan/tilt rotation, wherein the camera of the drone is rotated to follow the pan-tilt to obtain a second panoramic image. Specifically, adjusting the pitch angle of the UAV pan/tilt to the third pitch angle; controlling the gimbal to make a circumferential rotation by rotating the yaw angle of the gimbal, and acquiring the photographing device that is photographed during the process of following the motion rotation The third environment image; adjusting the pitch angle of the gimbal to the fourth pitch angle after one rotation of the control motion; controlling the pan-tilt to perform circumferential rotation by rotating the yaw angle of the gimbal, and acquiring the process of the camera rotating in following motion And capturing a fourth environment image; and splicing the third environment image and the fourth environment image to obtain a second panoramic image.

In one embodiment, acquiring a third feature point set of the third environment image and a fourth feature point set of the fourth environment image, performing feature point matching according to the third feature point set and the fourth feature point set to obtain the second target feature a point set, according to the second target feature point set, determining a second splicing line of the first environment image and the second environment image, and splicing the first environment image and the second environment image according to the second splicing line, to obtain the The second panoramic image.

It should be noted that, in the embodiment of the present invention, when the third environment image and the fourth environment image are spliced, the camera of the drone is calibrated, and the calibration value is used as the initial value of the parameter in the camera. Using the inertial measurement unit imu information as the initial value of the ba algorithm, the number of iterations can be reduced, and the relationship between the two characteristic images of the respective images can be used to perform matching of the targeted feature points, thereby reducing the matching error.

In one embodiment, the video processing device can control the rotation of the drone; wherein the camera of the drone follows the rotation of the drone to obtain the second panoramic image. Specifically, in the process of controlling the rotation of the drone, the video processing device may control the UAV to perform circumferential rotation in a clockwise direction, or control the UAV to perform circumferential rotation in a counterclockwise direction, The camera of the drone follows the rotation of the drone to obtain a second panoramic image.

It should be noted that the video processing device may select a plurality of location points on the flight path of the drone as a shooting point for capturing the panoramic image, so as to capture a plurality of video segments including the panoramic image, and the embodiment of the present invention shoots the panoramic image. The number of video segments is not limited.

S808. Generate a target video according to the first video segment, the second video segment, and the third video segment.

In an embodiment, after the target video is generated according to the first video segment, the second video segment, and the third video segment, if a play instruction for the target video is detected, the target video is played from the first video segment. Or, play the target video from the second video segment; or, play the target video from the third video segment. Specifically, in the process of playing the target video, the video processing device may start playing the target video from the first video segment, and pass the last image of the first video segment (ie, the first video segment during playback). The last video frame) gradually transitions to the second video segment to play, and gradually transitions to the third video segment playback through the last image of the second video segment (ie, the last video frame of the second video segment). Alternatively, in the process of playing the target video, the video processing device may start playing the target video from the second video segment, and gradually transition to the third video segment through the last video frame of the second video segment during the playing process. Play and gradually transition to the first video segment by the last image of the third video segment (ie, the last video frame of the third video segment). Alternatively, in the process of playing the target video, the video processing device may start playing the target video from the third video segment, and gradually transition to the second video segment through the last video frame of the third video segment during playback. Play, and gradually transition to the first video segment by the last image of the second video segment (ie, the last video frame of the second video segment).

In an embodiment, after the target video is generated according to the first video segment, the second video segment, and the third video segment, if the sharing operation instruction for the target video is detected, the target video is shared. For example, the video processing device may share the target video to a specified location of the application software when detecting an operation instruction for sharing the target video to application software (such as chat software such as WeChat, QQ, etc.). (such as WeChat friends, WeChat friends circle, QQ friends, etc.).

In the embodiment of the present invention, the video processing device may control the camera of the drone to capture the environment image at the first shooting point to obtain the first panoramic image, and generate the first video segment according to the first panoramic image; During the movement of the flight path from the first shooting point to the second shooting point and from the second shooting point to the third shooting point, the camera controlling the drone captures the second video segment; the drone arrives at the a third shooting point, controlling a shooting device of the drone to capture an environment image to obtain a second panoramic image, and generating a third video segment according to the second panoramic image; according to the first video segment, the second video segment, and the third The video segment generates a target video, and can generate a video including at least two panoramic video segments for the user in combination with the flight trajectory, and can realize a smooth transition between the panoramic video segment and the non-panoramic video segment, and realize a plurality of panoramic video segments. Smooth transition, can provide users with a global visual effect and local visual effects of a certain area, to meet the user's automation, intelligent shooting and video Needs.

Referring to FIG. 9, FIG. 9 is a schematic structural diagram of a video processing device according to an embodiment of the present invention. Specifically, the video processing device includes: a processor 901, a memory 902, a user interface 903, and a data interface 904, wherein the user interface 903 user outputs a target video.

The memory 902 may include a volatile memory; the memory 902 may also include a non-volatile memory; the memory 902 may also include a combination of the above types of memory. The processor 901 can be a central processing unit (CPU). The processor 901 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), or any combination thereof.

Optionally, the memory 902 is configured to store program instructions. The processor 901 can call the program instructions stored in the memory 902 for performing the following steps:

Optionally, the first shooting point, the second shooting point, and the third shooting point are three position points on the flight path.

Optionally, the first photographing point is a first position point selected according to the received selection operation on the flight path; the second photographing point is according to the received on the flight track. Selecting a second position point selected by the operation; the third shooting point is a third position point selected according to the received selection operation on the flight path.

Further, the processor 901 calls the program instructions stored in the memory 902 and is further configured to perform the following steps:

Obtaining a first location point, a second location point, and a third location point;

Determining, according to the location information of the first location point and the second location point, a first flight trajectory of the drone;

Determining, according to the location information of the second location point and the third location point, a second flight trajectory of the drone;

The first position point is used as the first shooting point, the second position point is used as the second shooting point, and the third position point is used as the third shooting point.

And generating a first linear trajectory from the first photographing point and the second photographing point according to the first trajectory generation rule according to the position information of the first position point and the second position point;

The first straight track is determined as the first flight trajectory.

And generating a second linear trajectory from the second photographing point and the third photographing point according to the first trajectory generation rule according to the position information of the second position point and the third position point;

The second linear trajectory is determined as a second flight trajectory.

And generating a first curved trajectory from the first photographing point and the second photographing point according to the second trajectory generation rule according to the position information of the first position point and the second position point;

The first curved trajectory is determined as the first flight trajectory.

And generating a second curved track from the third shooting point and the fourth shooting point according to the second track generation rule according to the second position point and the third position point position information;

The second curved trajectory is determined as a second flight trajectory.

The last image of the first panoramic image is used as the first video frame of the second video segment, and the last video frame of the second video segment is used as the first image of the second panoramic image.

Controlling the rotation of the drone or carrying the pan-tilt rotation of the photographing device, wherein the photographing device of the drone follows the drone or the pan-tilt rotates the photographing environment image to obtain the first panoramic image.

Controlling the rotation of the drone or carrying the pan/tilt rotation of the photographing device, wherein the photographing device of the drone follows the drone or the pan-tilt rotates the photographing environment image to obtain the second panoramic image.

Adjusting a pitch angle of the drone head to a first pitch angle;

Controlling the pan-tilt to perform circumferential rotation by rotating the yaw angle of the pan-tilt, and acquiring a first environment image captured by the photographing device during the following rotation of the motion;

Adjusting the pitch angle of the pan/tilt to a second pitch angle after controlling the one rotation of the motion;

Controlling the pan-tilt to perform circumferential rotation by rotating the yaw angle of the pan/tilt, and acquiring a second environment image captured by the photographing device during the following rotation of the motion;

And splicing the first environment image and the second environment image to obtain the first panoramic image.

Adjusting a pitch angle of the drone head to a third pitch angle;

Controlling the pan-tilt to perform circumferential rotation by rotating the yaw angle of the pan-tilt, and acquiring a third environment image captured by the photographing device during the following rotation of the motion;

Adjusting the pitch angle of the pan/tilt to a fourth pitch angle after controlling the one rotation of the motion;

Controlling the pan-tilt to perform circumferential rotation by rotating the yaw angle of the pan-tilt, and acquiring a fourth environment image captured by the photographing device during the following rotation of the motion;

And splicing the third environment image and the fourth environment image to obtain the second panoramic image.

Obtaining a first feature point set of the first environment image and a second feature point set of the second environment image;

Performing feature point matching according to the first feature point set and the second feature point set to obtain a first target feature point set;

Determining, according to the first target feature point set, a first stitching line of the first environment image and the second environment image;

And splicing the first environment image and the second environment image according to the first splicing line to obtain the first panoramic image.

Obtaining a third feature point set of the third environment image and a fourth feature point set of the fourth environment image;

Performing feature point matching according to the third feature point set and the fourth feature point set to obtain a second target feature point set;

Determining, according to the second target feature point set, a second splicing line of the first environment image and the second environment image;

And splicing the first environment image and the second environment image according to the second splicing line to obtain the second panoramic image.

Playing the target video from the first video segment if a play instruction for the target video is detected; or

Playing the target video from the second video segment; or,

The target video is played starting from the third video segment.

The target video is shared if a sharing operation instruction for the target video is detected.

The embodiment of the invention further provides a drone, comprising: a fuselage; a power system disposed on the fuselage for providing flight power; and a processor for controlling the camera of the drone at the first Shooting an environment image to obtain a first panoramic image, and obtaining a first video segment according to the first panoramic image; moving from the first shooting point to the second shooting point according to a flight path of the drone and from the During the movement of the second shooting point to the third shooting point, the shooting device controlling the drone captures a second video segment, and the drone reaches the third shooting point to control the drone The photographing device captures the environment image to obtain a second panoramic image, and generates a third video segment according to the second panoramic image, and generates a target according to the first video segment, the second video segment, and the third video segment video.

Further, the processor is further configured to perform the following steps:

Determining the first straight trajectory as the first flight trajectory;

The second linear trajectory is determined as a second flight trajectory.

Further, the processor is further configured to perform the following steps:

Determining the first curved trajectory as the first flight trajectory;

The second curved trajectory is determined as a second flight trajectory.

Further, the processor is further configured to perform the following steps:

Adjusting a pitch angle of the drone head to a first pitch angle;

Further, the processor is further configured to perform the following steps:

Adjusting a pitch angle of the drone head to a third pitch angle;

Further, the processor is further configured to perform the following steps:

Playing the target video from the second video segment; or,

The target video is played starting from the third video segment.

Further, the processor is further configured to perform the following steps:

For a specific implementation of the processor in the UAV, reference may be made to the video processing method in the embodiment corresponding to FIG. 7 or FIG. 8 , and details are not described herein again. Among them, the drone can be a four-rotor UAV, a six-rotor UAV, a multi-rotor UAV, and the like. The power system may include a motor, an ESC, a propeller, etc., wherein the motor is responsible for driving the aircraft propeller, and the ESC is responsible for controlling the speed of the motor of the aircraft.

The embodiment of the invention further provides a video processing system, comprising: a video processing device and a drone;

The video processing device is further configured to control the photographing device of the drone to capture an environment image at a first photographing point to obtain a first panoramic image, and obtain a first video segment according to the first panoramic image; Controlling the camera of the drone to capture a second video segment during movement of the flight path from the first shooting point to the second shooting point and from the second shooting point to the third shooting point When the drone arrives at the third shooting point, the camera of the drone is controlled to capture an environment image to obtain a second panoramic image, and a third video segment is generated according to the second panoramic image, according to the The first video segment, the second video segment, and the third video segment generate a target video.

Further, the video processing device is configured to acquire a first location point, a second location point, and a third location point; and determining, according to the location information of the first location point and the second location point, the unmanned a first flight trajectory of the machine; determining, according to the second location point and the location information of the third location point, a second flight trajectory of the drone; using the first location point as the first Shooting a point, using the second position point as the second shooting point, and using the third position point as the third shooting point.

Further, the video processing device is configured to generate, according to the first trajectory generation rule, the first photographic point and the second photographic point according to the location information of the first location point and the second location point. a first straight trajectory between the first trajectory determined as the first flight trajectory, and according to the position information of the second position point and the third position point, according to the first trajectory generation rule And generating a second straight line trajectory from the second photographing point and the third photographing point, and determining the second straight line locus as a second flight locus.

Further, the video processing device is further configured to generate, according to the second track generation rule, the first shooting point and the second shooting according to the location information of the first location point and the second location point. a first curve trajectory between the points, determining the first curve trajectory as the first flight trajectory, and generating a rule according to the second trajectory according to the position information of the second position point and the third position point, A second curved trajectory from the third shooting point and the fourth shooting point is generated, and the second curved trajectory is determined as a second flight trajectory.

Further, the video processing device is configured to use a last image of the first panoramic image as a first video frame of a second video segment, and a last video frame of the second video segment as the second video frame The first image of the panoramic image.

Further, the video processing device is configured to control the rotation of the drone or the pan/tilt rotation of the camera, wherein the camera of the drone follows the rotation of the drone or rotates the image of the environment The first panoramic image is obtained.

Further, the video processing device is configured to control the UAV to rotate or carry the pan/tilt rotation of the photographing device, wherein the photographing device of the drone follows the drone or the pan/tilt rotates The environmental image results in the second panoramic image.

Further, the video processing device is configured to adjust a pitch angle of the unmanned aerial platform to a first pitch angle; and by rotating a yaw angle of the pan/tilt, control the pan-tilt to perform a circumferential rotation, and acquire a first environment image captured by the photographing device during the following rotation of the motion; after controlling the motion to rotate for one week, adjusting a pitch angle of the pan/tilt to a second pitch angle, by rotating the pan/tilt a yaw angle, controlling the pan-tilt to perform a circumferential rotation, and acquiring a second environment image captured by the photographing device during the following rotation of the motion, and the first environment image and the second environment image A splicing process is performed to obtain the first panoramic image.

Further, the video processing device is configured to adjust a pitch angle of the unmanned aerial platform to a third pitch angle; by rotating a yaw angle of the pan/tilt, controlling the pan-tilt to perform a circumferential rotation, and acquiring a third environment image captured by the photographing device during the following rotation of the motion; adjusting the pitch angle of the pan/tilt to a fourth pitch angle after controlling the motion to rotate one revolution; by rotating the pan/tilt a yaw angle, controlling the pan-tilt to perform a circumferential rotation, and acquiring a fourth environment image captured by the photographing device during the following rotation of the motion; and the third environment image and the fourth environment image A splicing process is performed to obtain the second panoramic image.

Further, the video processing device is configured to acquire a first feature point set of the first environment image and a second feature point set of the second environment image, according to the first feature point set and the second Feature point set performing feature point matching to obtain a first target feature point set, and determining, according to the first target feature point set, a first stitching line of the first environment image and the second environment image, according to the first a splicing line, splicing the first environment image and the second environment image to obtain the first panoramic image.

Further, the video processing device is configured to acquire a third feature point set of the third environment image and a fourth feature point set of the fourth environment image, according to the third feature point set and the fourth Feature point set performing feature point matching to obtain a second target feature point set, and determining, according to the second target feature point set, a second stitching line of the first environment image and the second environment image, according to the second The splicing line is spliced to the first environment image and the second environment image to obtain the second panoramic image.

Further, the video processing device is further configured to: start playing the target video from the first video segment if a play instruction for the target video is detected; or start playing the video from the second video segment Target video; or, playing the target video from the third video segment.

Further, the video processing device is further configured to share the target video if a sharing operation instruction for the target video is detected.

Also provided in an embodiment of the present invention is a computer readable storage medium storing a computer program, which when executed by a processor, implements the implementation of FIG. 7 or FIG. 8 of the present invention. The video processing device of the embodiment of the present invention shown in FIG. 9 can also be implemented in the video processing method in the example, and details are not described herein again.

The computer readable storage medium may be an internal storage unit of the device described in any of the preceding embodiments, such as a hard disk or a memory of the device. The computer readable storage medium may also be an external storage device of the device, such as a plug-in hard disk equipped on the device, a smart memory card (SMC), and a secure digital (SD) card. , Flash Card, etc. Further, the computer readable storage medium may also include both an internal storage unit of the device and an external storage device. The computer readable storage medium is for storing the computer program and other programs and data required by the terminal. The computer readable storage medium can also be used to temporarily store data that has been output or is about to be output.

One of ordinary skill in the art can understand that all or part of the process of implementing the foregoing embodiments can be completed by a computer program to instruct related hardware, and the program can be stored in a computer readable storage medium. When executed, the flow of an embodiment of the methods as described above may be included. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

The above is only the preferred embodiment of the present invention, and the scope of the present invention is not limited thereto, and thus equivalent changes made in the claims of the present invention are still within the scope of the present invention.

Claims

A video processing method is applied to a drone, wherein the drone is configured with a photographing device, wherein the method comprises:

The photographing device that controls the drone captures an environment image at a first photographing point to obtain a first panoramic image, and obtains a first video segment according to the first panoramic image;

Controlling the shooting of the drone by the drone during the movement of the drone from the first photographing point to the second photographing point and from the second photographing point to the third photographing point Two video segments;

When the drone reaches the third shooting point, the camera of the drone is controlled to capture an environment image to obtain a second panoramic image, and a third video segment is generated according to the second panoramic image;

Generating a target video according to the first video segment, the second video segment, and the third video segment.
The method of claim 1 wherein the first photographing point, the second photographing point, and the third photographing point are three position points on the flight path.
Method according to claim 1 or 2, characterized in that

The first photographing point is a first position point selected according to the received selection operation on the flight locus;

The second photographing point is a second position point selected according to the received selection operation on the flight locus;

The third photographing point is a third position point selected according to the received selection operation on the flight locus.
The method according to any one of claims 1 to 3, wherein the flight path comprises a first flight trajectory and a second flight trajectory, wherein the method further comprises:

Obtaining a first location point, a second location point, and a third location point;

Determining, according to the location information of the first location point and the second location point, a first flight trajectory of the drone;

Determining, according to the location information of the second location point and the third location point, a second flight trajectory of the drone;

The first position point is used as the first shooting point, the second position point is used as the second shooting point, and the third position point is used as the third shooting point.
The method according to claim 4, wherein the determining the first flight trajectory of the drone according to the location information of the first location point and the second location point comprises:

And generating a first linear trajectory from the first photographing point and the second photographing point according to the first trajectory generation rule according to the position information of the first position point and the second position point;

Determining the first straight trajectory as the first flight trajectory;

Determining, according to the location information of the second location point and the third location point, the second flight trajectory of the drone, including:

And generating a second linear trajectory from the second photographing point and the third photographing point according to the first trajectory generation rule according to the position information of the second position point and the third position point;

The second linear trajectory is determined as a second flight trajectory.
The method according to claim 4, wherein the determining the first flight trajectory of the drone according to the location information of the first location point and the second location point comprises:

And generating a first curved trajectory from the first photographing point and the second photographing point according to the second trajectory generation rule according to the position information of the first position point and the second position point;

Determining the first curved trajectory as the first flight trajectory;

Determining, according to the location information of the second location point and the third location point, the second flight trajectory of the drone, including:

And generating a second curved track from the third shooting point and the fourth shooting point according to the second track generation rule according to the second position point and the third position point position information;

The second curved trajectory is determined as a second flight trajectory.
The method of claim 1 wherein

The last image of the first panoramic image is used as the first video frame of the second video segment, and the last video frame of the second video segment is used as the first image of the second panoramic image.
The method according to claim 1, wherein the controlling the shooting device of the drone to capture an environment image to obtain a first panoramic image comprises:

Controlling the rotation of the drone or carrying the pan-tilt rotation of the photographing device, wherein the photographing device of the drone follows the drone or the pan-tilt rotates the photographing environment image to obtain the first panoramic image.
The method according to claim 1, wherein the controlling the camera of the drone to capture an environment image to obtain a second panoramic image comprises:

Controlling the rotation of the drone or carrying the pan/tilt rotation of the photographing device, wherein the photographing device of the drone follows the drone or the pan-tilt rotates the photographing environment image to obtain the second panoramic image.
The method according to claim 8, wherein the controlling the camera of the drone to capture an environment image to obtain a first panoramic image comprises:

Adjusting a pitch angle of the drone head to a first pitch angle;

Controlling the pan-tilt to perform circumferential rotation by rotating the yaw angle of the pan-tilt, and acquiring a first environment image captured by the photographing device during the following rotation of the motion;

Adjusting the pitch angle of the pan/tilt to a second pitch angle after controlling the one rotation of the motion;

Controlling the pan-tilt to perform circumferential rotation by rotating the yaw angle of the pan/tilt, and acquiring a second environment image captured by the photographing device during the following rotation of the motion;

And splicing the first environment image and the second environment image to obtain the first panoramic image.
The method according to claim 9, wherein the controlling the camera of the drone to capture an environment image to obtain a second panoramic image comprises:

Adjusting a pitch angle of the drone head to a third pitch angle;

Controlling the pan-tilt to perform circumferential rotation by rotating the yaw angle of the pan-tilt, and acquiring a third environment image captured by the photographing device during the following rotation of the motion;

Adjusting the pitch angle of the pan/tilt to a fourth pitch angle after controlling the one rotation of the motion;

Controlling the pan-tilt to perform circumferential rotation by rotating the yaw angle of the pan-tilt, and acquiring a fourth environment image captured by the photographing device during the following rotation of the motion;

And splicing the third environment image and the fourth environment image to obtain the second panoramic image.
The method according to claim 10, wherein the splicing the first environment image and the second environment image to obtain the first panoramic image comprises:

Obtaining a first feature point set of the first environment image and a second feature point set of the second environment image;

Performing feature point matching according to the first feature point set and the second feature point set to obtain a first target feature point set;

Determining, according to the first target feature point set, a first stitching line of the first environment image and the second environment image;

And splicing the first environment image and the second environment image according to the first splicing line to obtain the first panoramic image.
The method according to claim 11, wherein the splicing process of the third environment image and the fourth environment image to obtain the second panoramic image comprises:

Obtaining a third feature point set of the third environment image and a fourth feature point set of the fourth environment image;

Performing feature point matching according to the third feature point set and the fourth feature point set to obtain a second target feature point set;

Determining, according to the second target feature point set, a second splicing line of the first environment image and the second environment image;

And splicing the first environment image and the second environment image according to the second splicing line to obtain the second panoramic image.
The method according to any one of claims 1 to 13, wherein after the generating the target video according to the first video segment, the second video segment and the third video segment, the method further comprises:

Playing the target video from the first video segment if a play instruction for the target video is detected; or

Playing the target video from the second video segment; or,

The target video is played starting from the third video segment.
The method according to any one of claims 1 to 13, wherein after the generating the target video according to the first video segment, the second video segment and the third video segment, the method further comprises:

The target video is shared if a sharing operation instruction for the target video is detected.
A video processing device, comprising: a memory and a processor;

The memory is configured to store program instructions;

The processor executes program instructions stored in the memory, and when the program instructions are executed, the processor is configured to perform the following steps:

The photographing device that controls the drone captures an environment image at a first photographing point to obtain a first panoramic image, and obtains a first video segment according to the first panoramic image;

Controlling the shooting of the drone by the drone during the movement of the drone from the first photographing point to the second photographing point and from the second photographing point to the third photographing point Two video segments;

When the drone reaches the third shooting point, the camera of the drone is controlled to capture an environment image to obtain a second panoramic image, and a third video segment is generated according to the second panoramic image;

Generating a target video according to the first video segment, the second video segment, and the third video segment.
The apparatus according to claim 16, wherein said first photographing point, said second photographing point, and said third photographing point are three position points on said flight locus.
Device according to claim 16 or 17, characterized in that

The first photographing point is a first position point selected according to the received selection operation on the flight locus;

The second photographing point is a second position point selected according to the received selection operation on the flight locus;

The third photographing point is a third position point selected according to the received selection operation on the flight locus.
Apparatus according to any of claims 16-18, wherein

The flight trajectory includes a first flight trajectory and a second flight trajectory, and the processor is configured to acquire a first location point, a second location point, and a third location point; according to the first location point and the second location Positioning information of the location point, determining a first flight trajectory of the drone; determining a second flight trajectory of the drone according to location information of the second location point and the third location point; The first position point is used as the first shooting point, the second position point is used as the second shooting point, and the third position point is used as the third shooting point.
The device according to claim 19, characterized in that

The processor is configured to generate, according to the first location point and the second location point, a first trajectory generation rule, between the first shooting point and the second shooting point Determining the first straight trajectory as the first flight trajectory, and generating a slave according to the first trajectory generation rule according to the position information of the second position point and the third position point a second linear trajectory between the second photographing point and the third photographing point; the second straight trajectory is determined as a second flight trajectory.
The device according to claim 19, characterized in that

The processor is configured to generate, according to the second location generation rule, the second trajectory generation rule according to the location information of the first location point and the second location point, between the first shooting point and the second shooting point a first curve trajectory; determining the first curve trajectory as the first flight trajectory; generating a rule from the second trajectory according to position information of the second position point and the third position point A second curved trajectory between the third shooting point and the fourth shooting point determines the second curved trajectory as a second flight trajectory.
The device of claim 16 wherein:

The processor is configured to use a last image of the first panoramic image as a first video frame of a second video segment, and a last video frame of the second video segment as the second panoramic image The first image.
The device of claim 16 wherein:

The processor is configured to control the rotation of the drone or to carry the pan/tilt rotation of the photographing device, wherein the photographing device of the drone follows the drone rotation or the pan/tilt rotates the photographing environment image to obtain the The first panoramic image.
The device of claim 16 wherein:

The processor is configured to control the UAV to rotate or carry the pan/tilt rotation of the photographing device, wherein the photographing device of the drone follows the rotation of the drone or the pan/tilt rotates to capture an image of the environment The second panoramic image.
The device according to claim 23, wherein

The processor is configured to adjust a pitch angle of the unmanned aerial platform to a first pitch angle; control a rotation of the pan/tilt by rotating a yaw angle of the pan/tilt, and acquire the camera And capturing a first environment image during the following rotation of the motion; adjusting a pitch angle of the pan/tilt to a second pitch angle after controlling the rotation of the motion; and rotating the yaw angle of the pan/tilt Controlling the pan-tilt to perform circumferential rotation, and acquiring a second environment image captured by the photographing device during the following rotation of the motion; splicing the first environment image and the second environment image And obtaining the first panoramic image.
The device according to claim 24, wherein

The processor is configured to adjust a pitch angle of the UAV pan/tilt to a third pitch angle; control the pan-tilt to perform circumferential rotation by rotating a yaw angle of the pan-tilt, and acquire the photographing device a third environment image captured during the following rotation of the motion; adjusting the pitch angle of the pan/tilt to a fourth pitch angle after controlling the rotation of the motion; and rotating the yaw angle of the pan/tilt Controlling the pan-tilt to perform circumferential rotation, and acquiring a fourth environment image captured by the photographing device during the following rotation of the motion; splicing the third environment image and the fourth environment image And obtaining the second panoramic image.
The device according to claim 25, wherein

The processor is configured to acquire a first feature point set of the first environment image and a second feature point set of the second environment image, according to the first feature point set and the second feature point set Performing feature point matching to obtain a first target feature point set, and determining, according to the first target feature point set, a first stitching line of the first environment image and the second environment image, according to the first stitching line, The first environment image and the second environment image are spliced to obtain the first panoramic image.
The device according to claim 26, wherein

The processor is configured to acquire a third feature point set of the third environment image and a fourth feature point set of the fourth environment image, according to the third feature point set and the fourth feature point set Performing feature point matching to obtain a second target feature point set, and determining, according to the second target feature point set, a second splicing line of the first environment image and the second environment image, according to the second splicing line, The first environment image and the second environment image are spliced to obtain the second panoramic image.
Apparatus according to any one of claims 16-28, wherein

The processor is further configured to: start playing the target video from the first video segment if a play instruction for the target video is detected; or start playing the target video from the second video segment; Or, playing the target video from the third video segment.
Apparatus according to any one of claims 16-28, wherein

The processor is further configured to share the target video if a sharing operation instruction for the target video is detected.
A drone, characterized in that it comprises:

body;

a power system disposed on the fuselage for providing flight power;

a processor, configured to control the camera of the drone to capture an environment image at a first shooting point to obtain a first panoramic image, and obtain a first video segment according to the first panoramic image, and follow the flight path in the drone Controlling the camera of the drone to capture a second video segment during the movement from the first photographing point to the second photographing point and from the second photographing point to the third photographing point The drone reaches the third shooting point, controls the shooting device of the drone to capture the environment image to obtain a second panoramic image, and generates a third video segment according to the second panoramic image, according to the first video segment. And generating, by the second video segment and the third video segment, a target video.
The drone according to claim 31, wherein

The processor is configured to acquire a first location point, a second location point, and a third location point, and determine, according to the location information of the first location point and the second location point, the first a flight trajectory, determining a second flight trajectory of the drone according to the second location point and the location information of the third location point, and using the first location point as the first shooting point, The second position point is used as the second shooting point, and the third position point is used as the third shooting point.
The drone according to claim 31, wherein

The processor is configured to control the rotation of the drone or to carry the pan/tilt rotation of the photographing device, wherein the photographing device of the drone follows the drone rotation or the pan/tilt rotates the photographing environment image to obtain the The first panoramic image.
The drone according to claim 31, wherein

The processor is configured to control the UAV to rotate or carry the pan/tilt rotation of the photographing device, wherein the photographing device of the drone follows the rotation of the drone or the pan/tilt rotates to capture an image of the environment The second panoramic image.
The drone according to claim 33, wherein

The processor is configured to adjust a pitch angle of the unmanned aerial platform to a first pitch angle; control a rotation of the pan/tilt by rotating a yaw angle of the pan/tilt, and acquire the camera And capturing a first environment image during the following rotation of the motion; adjusting a pitch angle of the pan/tilt to a second pitch angle after controlling the rotation of the motion; and rotating the yaw angle of the pan/tilt Controlling the pan-tilt to perform circumferential rotation, and acquiring a second environment image captured by the photographing device during the following rotation of the motion; splicing the first environment image and the second environment image And obtaining the first panoramic image.
The drone according to claim 34, characterized in that

The processor is configured to adjust a pitch angle of the UAV pan/tilt to a third pitch angle; control the pan-tilt to perform circumferential rotation by rotating a yaw angle of the pan-tilt, and acquire the photographing device a third environment image captured during the following rotation of the motion; adjusting the pitch angle of the pan/tilt to a fourth pitch angle after controlling the rotation of the motion; and rotating the yaw angle of the pan/tilt Controlling the pan-tilt to perform circumferential rotation, and acquiring a fourth environment image captured by the photographing device during the following rotation of the motion; splicing the third environment image and the fourth environment image And obtaining the second panoramic image.
The drone according to claim 35, characterized in that

The processor is configured to acquire a first feature point set of the first environment image and a second feature point set of the second environment image, according to the first feature point set and the second feature point set Performing feature point matching to obtain a first target feature point set, and determining, according to the first target feature point set, a first stitching line of the first environment image and the second environment image, according to the first stitching line, The first environment image and the second environment image are spliced to obtain the first panoramic image.
The drone according to claim 36, characterized in that

The processor is configured to acquire a third feature point set of the third environment image and a fourth feature point set of the fourth environment image, according to the third feature point set and the fourth feature point set Performing feature point matching to obtain a second target feature point set, and determining, according to the second target feature point set, a second splicing line of the first environment image and the second environment image, according to the second splicing line, The first environment image and the second environment image are spliced to obtain the second panoramic image.
A video processing system, comprising: a video processing device and a drone;

The video processing device is configured to send a flight control instruction to the aircraft, where the flight control instruction is used to instruct the drone to fly according to the determined flight trajectory;

The drone is configured to control the drone to fly according to the flight trajectory and control a photographing device mounted on the drone to perform shooting according to the flight control instruction;

The video processing device is further configured to control the camera of the drone to capture an environment image at a first shooting point to obtain a first panoramic image, and obtain a first video segment according to the first panoramic image, Controlling the camera of the drone to capture a second video segment during movement of the flight path from the first shooting point to the second shooting point and from the second shooting point to the third shooting point When the drone arrives at the third shooting point, the camera of the drone is controlled to capture an environment image to obtain a second panoramic image, and a third video segment is generated according to the second panoramic image, according to the The first video segment, the second video segment, and the third video segment generate a target video.
The system of claim 39, wherein

The video processing device is configured to acquire a first location point, a second location point, and a third location point, and determine, according to the location information of the first location point and the second location point, the unmanned aerial vehicle Determining, according to the second position point and the position information of the third position point, a second flight trajectory of the drone, using the first position point as the first shooting point The second position point is used as the second shooting point, and the third position point is used as the third shooting point.
The system of claim 39, wherein

The video processing device is configured to control the rotation of the drone or to carry the pan/tilt rotation of the photographing device, wherein the photographing device of the drone follows the rotation of the drone or rotates the image of the environment to obtain an image of the environment The first panoramic image is described.
The system of claim 39, wherein

The video processing device is configured to control the UAV to rotate or carry the pan/tilt rotation of the photographing device, wherein the photographing device of the drone follows the drone rotation or the pan/tilt rotates to capture an environment image The second panoramic image is obtained.
The system of claim 41, wherein

The video processing device is configured to adjust a pitch angle of the unmanned aerial platform to a first pitch angle; control a rotation of the pan/tilt by rotating a yaw angle of the pan/tilt, and acquire the shooting a first environment image captured by the device during the following rotation of the motion; adjusting the pitch angle of the pan/tilt to a second pitch angle after controlling the rotation of the motion; and rotating the yaw of the pan/tilt An angle, controlling the pan-tilt to perform circumferential rotation, and acquiring a second environment image captured by the photographing device during the following rotation of the motion; splicing the first environment image and the second environment image Processing to obtain the first panoramic image.
The system of claim 42 wherein:

The video processing device is configured to adjust a pitch angle of the drone head to a third pitch angle; control the pan head to perform a circumferential rotation by rotating a yaw angle of the pan/tilt, and acquire the photographing a third environment image captured by the device during the following rotation of the motion; adjusting the pitch angle of the pan/tilt to a fourth pitch angle after controlling the rotation of the motion; and rotating the yaw of the pan/tilt a corner, controlling the pan-tilt to perform a circumferential rotation, and acquiring a fourth environment image captured by the photographing device during the following rotation of the motion; splicing the third environment image and the fourth environment image Processing to obtain the second panoramic image.
The system of claim 43 wherein:

The video processing device is configured to acquire a first feature point set of the first environment image and a second feature point set of the second environment image, according to the first feature point set and the second feature point And performing a feature point matching to obtain a first target feature point set, and determining, according to the first target feature point set, a first splicing line of the first environment image and the second environment image, according to the first splicing line And splicing the first environment image and the second environment image to obtain the first panoramic image.
The system of claim 44, wherein

The video processing device is configured to acquire a third feature point set of the third environment image and a fourth feature point set of the fourth environment image, according to the third feature point set and the fourth feature point And performing a feature point matching to obtain a second target feature point set, and determining, according to the second target feature point set, a second splicing line of the first environment image and the second environment image, according to the second splicing line And splicing the first environment image and the second environment image to obtain the second panoramic image.
A system according to any of claims 39-46, wherein

The video processing device is further configured to: start playing the target video from the first video segment if a play instruction for the target video is detected; or start playing the target video from the second video segment Or, the target video is played starting from the third video segment.
A system according to any of claims 39-46, wherein

The video processing device is further configured to share the target video if a sharing operation instruction for the target video is detected.
A computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the method of any one of claims 1 to 15.