WO2020014953A1

WO2020014953A1 - Image processing method and device

Info

Publication number: WO2020014953A1
Application number: PCT/CN2018/096410
Authority: WO
Inventors: 李阳
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2018-07-20
Filing date: 2018-07-20
Publication date: 2020-01-23
Also published as: CN110720209A; CN110720209B

Abstract

Provided in an embodiment of the present invention are an image processing method and device. The image processing method can be applied in an unmanned aerial vehicle system comprising a photographing device, and comprises: acquiring a positional relationship between a photographic object and an unmanned aerial vehicle; controlling flight of the unmanned aerial vehicle according to a control rule, wherein the control rule is used to maintain the positional relationship between the unmanned aerial vehicle and the photographic object; and using a photographing device to photograph the photographic object so as to obtain multiple image data items wherein the multiple image data items are used to generate a stop-motion animation at a preset frame rate. Compared with existing hand-held devices, the device provided in the embodiment of the present invention can produce a stop-motion animation consisting of stable images.

Description

Image processing method and equipment

Technical field

The present invention relates to the field of image processing technology, and in particular, to an image processing method and device.

Background technique

Stop motion animation is to shoot an object frame by frame and then project the captured images continuously to produce an animation effect. At present, freeze-frame animation is shot by a handheld device, and the shooting angle is limited by the position of the photographer, which results in insufficient freedom of the picture.

In addition, it has been found in practice that the shooting of stop motion animation needs to keep the relative positional relationship between the shooting target and the camera stable. Therefore, shooting a stop motion animation by a handheld device will result in a tedious shooting process and poor stability of the shooting screen.

At present, with the development of flight technology, drones or drones have become more popular research topics at present. Therefore, how to shoot freeze-frame animation based on drones is an urgent problem to be solved.

Summary of the invention

Embodiments of the present invention provide an image processing method and device, which can obtain a stop motion animation with a more stable picture.

In a first aspect, an embodiment of the present invention provides an image processing method, which is applied to a drone system including a photographing device, and the method includes:

Obtain the positional relationship between the subject and the drone;

Controlling the drone flying based on a control rule, the control rule is used to control the drone to maintain the position relationship with the photographic subject;

And photographing the photographic subject through a photographing device to obtain a plurality of image data, where the plurality of image data is used to generate a stop motion animation at a preset frame rate.

In a second aspect, an embodiment of the present invention provides an image processing device, which is applied to a drone system including a photographing device, and the image processing device includes a memory, a controller, and a photographing device;

The memory is used to store program instructions;

The controller is configured to execute a program instruction stored in the memory. When the program instruction is executed, the controller is configured to perform the following steps:

Obtain the positional relationship between the subject and the drone;

Controlling the drone flying based on a control rule, which is used to maintain the positional relationship with the photographic subject;

A plurality of image data are obtained by photographing the photographed subject through the photographing device, and the plurality of image data is used to generate a stop motion animation at a preset frame rate.

According to a third aspect, an embodiment of the present invention provides a drone, including a body and a shooting device of the drone;

The photographing device is mounted on the body of the drone, or is provided on a gimbal, and is mounted on the body of the drone through the gimbal;

The drone further includes a controller and a memory, and the controller is configured to call code stored in the memory to execute the image processing method according to the first aspect.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where the computer storage medium stores a computer program, the computer program includes program instructions, and the program instructions, when executed by a controller, cause the control The processor executes the image processing method according to the first aspect.

In a fifth aspect, an embodiment of the present invention provides a computer program product. The computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to make a computer implement the first aspect The image processing method.

In the embodiment of the present invention, the drone can be controlled to maintain the positional relationship with the subject based on control rules, and the subject is photographed by a photographing device to obtain multiple pictures or multiple Video to generate stop motion animation, which makes the subject more stable in the picture, and enriches the shooting perspective of stop motion animation.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly explain the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some of the present invention. For those of ordinary skill in the art, other embodiments may be obtained based on these drawings without paying creative labor.

FIG. 1 is a schematic structural diagram of an unmanned aerial vehicle system according to an embodiment of the present invention;

2 is a schematic structural diagram of another drone system according to an embodiment of the present invention;

3 is a schematic flowchart of an image processing method according to an embodiment of the present invention;

4 is a schematic structural diagram of an image processing device according to an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a drone provided by an embodiment of the present invention.

detailed description

Hereinafter, some embodiments of the present invention will be described in detail with reference to the drawings. In the case of no conflict, it is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention. In addition, in the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms "a," "the," and "the" as used in this invention and in the claims are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be understood that the term "and / or" as used herein means any or all possible combinations that include one or more of the associated listed items.

The image processing method provided by the embodiment of the present invention is applied to a drone system including a shooting device. The drone system may include a drone, a control device, a display device, and a shooting device. The control device and the display device may be integrated. Become a device, or separate devices. The shooting device can be directly mounted on the body or main body of the drone, or it can be mounted on the drone through the gimbal.

In the embodiment of the present invention, the control device may be a control terminal of a drone, and specifically may be a remote controller, a smart phone, a tablet computer, a laptop computer, a ground station, a wearable device (watch, bracelet) One or more. The display device is used to display a picture or video taken by the drone, and to display the stop motion animation generated. The control device and the display device may be located in the same casing, that is, integrated into the same device, or may be physically separated devices, that is, belong to different devices.

In the embodiment of the present invention, the drone may be a rotary wing drone, such as a quad-rotor drone, a six-rotor drone, an eight-rotor drone, or a fixed-wing drone. The UAV includes a power system, which is used to provide flying power for the UAV. The power system includes one or more of a propeller, a motor, and an ESC. It can be understood that the drone may also be an unmanned vehicle or an unmanned ship.

In the embodiment of the present invention, the photographing device may be directly mounted on the body or main body of the drone, or may be mounted on the body or main body of the drone through the gimbal. The shooting device is used for shooting image data during the flight of the drone. The shooting device includes, but is not limited to, a multi-spectral imager, a hyper-spectral imager, a visible light camera, and an infrared camera. In the stabilization system, the gimbal motor compensates the shooting angle of the shooting device by adjusting the rotation angle of the rotating shaft, and prevents or reduces the shake of the shooting device by setting an appropriate buffer mechanism.

The image data includes pictures or videos.

Please refer to FIG. 1. FIG. 1 is a schematic structural diagram of a drone system according to an embodiment of the present invention. The drone 12 can execute the image processing method according to the embodiment of the present invention. The drone 12 is provided with a photographing device 121. The photographing device 121 is arranged on the gimbal 122, and the gimbal 122 is mounted on the main body of the drone 12. The power system of the drone 12 uses the propeller 123 as an example, and the control device 11 can operate the drone 12 flight.

As shown in FIG. 2, FIG. 2 is a schematic structural diagram of another drone system provided by an embodiment of the present invention. In FIG. 2, the drone 22 may also execute the image processing method according to the embodiment of the present invention. The drone 22 is also provided with a photographing device 221, wherein the photographing device 221 provided by the drone 22 is directly mounted on the main body of the drone 22. Among them, the power system of the drone 22 uses the propeller 222 as an example.

Among them, in the drone system shown in FIG. 1 and FIG. 2, the control device 11 takes an integrated display device as an example, and can display a plurality of pictures or videos taken, and can also display a generated image at a preset frame rate. Stop motion animation.

Among them, the difference between the drone system shown in FIG. 1 and FIG. 2 is that the shooting device shown in FIG. 1 is set on the gimbal, and is mounted on the body of the drone through the gimbal, so based on the control rules When controlling the drone flight to maintain the positional relationship between the drone and the subject, it can also be achieved by adjusting the attitude of the gimbal. The photographing device shown in FIG. 2 is directly mounted on the body of the drone, so when maintaining the positional relationship between the drone and the subject, in addition to controlling the flying attitude of the drone, you can also adjust the shooting parameters by Focus or attitude of the drone to maintain the positional relationship between the drone and the subject.

In the embodiment of the present invention, the drone 12 in FIG. 1 or the drone 22 in FIG. 2 can obtain the positional relationship between the shooting object and the drone, and control the drone to fly based on the control rule to maintain Positional relationship with the photographic subject, and photographing the photographic subject through the photographing device to obtain a plurality of image data, so that a stop motion animation can be generated at a preset frame rate based on the plurality of image data.

The drone may send the plurality of image data to a control device, such as the control device 11 in FIG. 1 or the control device 21 in FIG. 2, and generate a stop motion animation at a preset frame rate based on the plurality of image data. Among them, the control device 11 in FIG. 1 or the control device 21 in FIG. 2 may be configured with an image processing device and / or a display device.

In the embodiment of the present invention, the control rule may be a control rule set by a user through a control device, such as a remote control device; and may also be a control rule output by a drone based on the detection of the surrounding environment by a sensor. This embodiment of the present invention does not do limited. Based on the control rule, the drone is controlled to maintain the positional relationship between the drone and the subject, thereby making the subject more stable in the shooting picture.

For example, in one example, a drone has a vision sensor, which can observe the surrounding environment in depth and obtain depth information to determine the positional relationship between the subject and the drone. Therefore, when the position of the subject changes , The drone can compensate its positional relationship with the subject, for example, make corresponding compensation from the height and / or horizontal distance, so as to maintain the positional relationship with the subject, so that the The position of the subject remains relatively stable, making the shooting effect of the stop motion animation more stable and smooth.

In another example, the positional relationship between the subject and the drone can also be maintained in other ways, for example, the control device maintains the relationship between the subject and the drone by adjusting the flying attitude of the drone. Positional relationship, or adjust the attitude of the PTZ to maintain the positional relationship between the subject and the drone.

In yet another example, the position of the object photographed by the photographing device in the frame can be kept stable by adjusting the photographing parameters. For example, focus is adjusted to keep the size and position of the subject in the frame the same.

It can be understood that in other ways, the drone can also use vision technology to maintain the same size and position of the subject in the picture so as to maintain the positional relationship between the subject and the drone. Manual control equipment is needed to maintain the positional relationship between the subject and the drone.

Therefore, the embodiment of the present invention adopts a drone system to shoot a stop motion animation, which can ensure the stability of the subject in the shooting frame compared with the conventional method of shooting a stop motion animation by using a handheld device; meanwhile, the embodiment of the present invention Compared with the limitation of the shooting position of the handheld device due to the position of the photographer, various shooting angles can also be adopted, which enriches the shooting angle, such as a bird's-eye view, to take pictures or videos required for stop motion animation.

In the embodiment of the present invention, the image processing method may be executed under a combination of multiple modes. For example, smart mode, non-smart mode, sports mode, fixed mode, manual shooting mode, and automatic shooting mode.

Among them, the intelligent mode means that the drone or the gimbal on the drone can follow the subject. That is, when the position of the subject changes, the drone automatically controls the drone or the attitude of the drone also changes, so as to maintain the same positional relationship with the subject. For example, when the subject moves to the left of the shooting screen, the drone itself controls the drone or the drone controls the attitude change of the gimbal, so that the subject is still in the middle of the shooting screen. Non-intelligent mode means that the drone or the gimbal on the drone will not follow the subject. To maintain the position relationship between the drone and the subject, the user needs to manually operate the control device to remotely control the drone or PTZ attitude. For example, when the subject moves to the left of the shooting screen, the user needs to operate an interactive device such as a joystick on the control device to adjust the attitude of the drone or the PTZ so that the subject is still in the middle of the shooting screen.

Among them, the motion mode refers to the drone in the flight process, the drone takes a picture or video required for stop motion animation through the shooting device. In this way, when you need to control the position relationship between the drone and the subject, you can The trajectory of the subject is used to determine the flight trajectory. Fixed mode means that when the drone is in the hovering state, the drone or the control device takes the picture or video required for the stop motion animation through the shooting device. In this way, the drone can be controlled by adjusting the shooting parameters or the attitude of the gimbal Maintain the positional relationship with the subject.

Among them, the manual shooting mode means that when shooting a subject with shooting parameters, the user needs to shoot manually; while in the automatic shooting mode, the drone can automatically shoot at preset time intervals through the shooting device. In actual implementation, the user can send an automatic shooting instruction to the drone through the control device, so that the drone can take multiple pictures or multiple short videos at preset time intervals through the shooting device in the automatic shooting mode.

In the embodiment of the present invention, the image processing method may be executed under a combination of multiple modes. For example, the intelligent mode may be combined with the motion mode or the fixed mode, respectively, and the image processing method according to the embodiment of the present invention is executed in the intelligent motion mode or the intelligent fixed mode. Correspondingly, the non-intelligent mode may be combined with the motion mode or the fixed mode, respectively, and the image processing method according to the embodiment of the present invention is executed in the non-intelligent motion mode or the non-intelligent fixed mode. For another example, the manual shooting mode may be combined with a sports mode or a fixed mode, respectively, and the image processing method according to the embodiment of the present invention is performed in the sports mode or the fixed mode. Correspondingly, the automatic shooting mode may be combined with the sports mode or the fixed mode, respectively, and the image processing method according to the embodiment of the present invention is executed in the sports mode or the fixed mode.

The following further describes the image processing method provided by the embodiment of the present invention based on the UAV system shown above. Please refer to FIG. 3. FIG. 3 is a schematic flowchart of an image processing method according to an embodiment of the present invention. The image processing method shown in FIG. 3 may be applied to a drone. The image processing method may include the following steps:

101. Obtain a positional relationship between a shooting object and a drone;

In the embodiment of the present invention, obtaining the positional relationship between the shooting object and the drone includes: obtaining a shooting picture selected by a user, and obtaining depth information of the shooting object relative to the shooting device or the drone based on a visual sensor; based on The depth information acquires a positional relationship between the drone and a subject in the shooting frame. The shooting picture selected by the user may specifically be a shooting picture when the user controls the first picture taken by the shooting device through the control device.

102. Control the drone flying based on a control rule, where the control rule is used to control the drone to maintain the positional relationship with the photographic subject;

In an optional implementation manner, the control rule is specifically configured to control the drone to maintain the positional relationship with the photographed object while the drone is in flight. In this way, when the subject moves, the flight trajectory or attitude of the drone can be adjusted to maintain the positional relationship with the subject.

In another optional implementation manner, the control rule is specifically configured to control the drone to maintain the position relationship with the photographed object when the drone is in a hovering state. In this way, when the photographic subject moves, at least one of the posture of the photographing device, the photographing parameters, and the posture of the gimbal equipped with the photographing device can be controlled, so that the drone maintains the positional relationship with the photographed object.

In yet another optional embodiment, controlling the drone's flight based on control rules includes: receiving a follow mode instruction; and when the follow mode instruction is a follow mode, adjusting the attitude of the drone, the At least one of a posture of the photographing device and a posture of the pan / tilt where the photographing device is located to follow the photographed object. The following mode may also be referred to as the intelligent following mode described above.

In yet another optional embodiment, controlling the drone's flight based on control rules includes: receiving a follow mode instruction; when the follow mode instruction is a non-follow mode, the drone needs to receive a user input through a control device To adjust the flying attitude of the drone or the attitude of the gimbal to follow the subject. The non-following mode may also be referred to as the non-intelligent following mode described above.

Wherein, the above-mentioned four embodiments may be combined with each other, and the position relationship between the drone and the photographic subject is maintained in the following mode or the non-following mode when the drone is in flight; In hover state, maintain the positional relationship between the drone and the subject in follow mode or non-follow mode. The embodiment of the present invention is not limited.

103. Shoot the subject through a shooting device to obtain multiple pictures or multiple videos.

In an optional implementation manner, shooting the subject through a shooting device to obtain multiple pictures or videos includes: receiving a shooting instruction input by a user; responding to the shooting instruction, using the shooting device Shooting the subject to obtain multiple pictures or multiple videos. Among them, each picture corresponds to a shooting instruction.

In another optional implementation manner, shooting the subject to obtain multiple pictures includes: shooting the subject through the shooting device at a preset time interval to obtain multiple pictures. Among them, the drone or the shooting device can perform automatic shooting at preset time intervals when receiving an automatic shooting instruction. The image processing method may further include: acquiring a shooting parameter set by the shooting device. Correspondingly, in 103, the shooting object may be shot by the shooting device with the shooting parameter to obtain multiple pictures or multiple videos. The set shooting parameters may be: the shooting parameters set by the user each time a shooting instruction is entered in the manual shooting mode; or the shooting parameters set by the user in advance in the automatic shooting mode; or, according to the user's selection The shooting parameters determined by the shooting screen. In an implementation manner, the shooting parameter and the position relationship may be correspondingly stored in a database.

Correspondingly, in 101, obtaining the positional relationship between the photographic subject and the drone includes: obtaining the positional relationship between the photographic subject and the drone from a database and shooting parameters. This embodiment can realize the shooting connection of multiple flights. For example, after a user obtains multiple pictures of a subject according to the position relationship and shooting parameters in scene A, the user may obtain multiple pictures of the subject according to the position relationship and shooting parameters in scene B. By taking flying pictures of drones in multiple scenarios, and taking multiple pictures or videos with the same positional relationship or preset amplitude shooting parameters, the expressiveness of the generated stop motion animation is richer.

In another optional implementation manner, when shooting is performed with a shooting instruction or a preset time interval input by a user, shooting parameters may be adjusted by a preset range; and the shooting object is shot based on the adjusted shooting parameters. Get multiple pictures or videos. The shooting parameter may be at least one of a filter, a color style, a special effect, a focal length, a shutter, an aperture, and a sensitivity value ISO. In this way, the captured multiple pictures or videos can have certain changes, making the final stop motion animation more exciting.

In the embodiment of the present invention, the multiple pictures or videos are used to generate a stop motion animation at a preset frame rate. The preset frame rate may be set by a user through a control device or a display device, or may be sent to a control device or a drone by using other controls or instructions. The control device or the drone may send the multiple frames A picture or multiple videos generates a stop motion animation at a preset frame rate, where the control device can receive multiple pictures or multiple videos taken by the drone.

It can be seen that the embodiment of the present invention can control the drone to maintain the positional relationship with the photographic subject, and photograph the photographic subject through a photographing device to obtain multiple pictures or multiple videos, so that the photographing The subject is more stable in the frame and enriches the shooting angle of the stop motion animation.

Please refer to FIG. 4. FIG. 4 is a schematic structural diagram of an image processing device according to an embodiment of the present invention. The image processing device may be set in a drone. As shown in FIG. 4, the image processing device may include:

An acquisition module 301, configured to acquire a positional relationship between a shooting object and a drone;

A control module 302 is configured to control the drone flying based on a control rule, where the control rule is used to control the drone to maintain the position relationship with the photographic subject, and The subject is photographed to obtain multiple pictures or multiple videos.

In an optional implementation manner, the image processing device may further include a generating module configured to generate a stop motion animation at a preset frame rate based on the multiple pictures or multiple videos. In another optional implementation manner, the image processing device may further include a communication module, which is configured to send the multiple pictures or videos to the control device, and the control device generates a freeze frame at a preset frame rate. Animation.

In an optional implementation manner, the obtaining module 301 obtains the positional relationship between the photographic subject and the drone, which is specifically:

Acquiring a shooting picture selected by a user, and acquiring depth information of a shooting object in the shooting picture relative to the shooting device or the drone based on a visual sensor;

Based on the photographed picture and the depth information, a positional relationship between the drone and the photographed object is acquired.

In an optional implementation manner, the control rule is specifically configured to control the drone to maintain the positional relationship with the photographed object while the drone is in flight;

Alternatively, when the drone is in a hovering state, control the drone to maintain the positional relationship with the shooting object.

In an optional implementation manner, the control module 302 photographs the photographed object through a photographing device to obtain multiple pictures or multiple videos, specifically:

Receiving shooting instructions input by the user;

In response to the shooting instruction, shooting the subject through the shooting device to obtain multiple pictures or multiple videos.

At a preset time interval, the photographing subject is photographed by the photographing device to obtain multiple pictures or multiple videos.

In an optional implementation manner, the control module 302 controls the drone flight based on a control rule, specifically:

Receiving a follow mode command;

When the following mode instruction is a following mode, adjust at least one of a posture of the drone, a posture of the photographing device, and a posture of a pan / tilt where the photographing device is located to follow the photographic subject. .

The positional relationship and shooting parameters between the photographic object and the drone are obtained from a database, and the positional relationship and the shooting parameters required for shooting a stop motion animation are stored in the database in advance.

Correspondingly, the control module 302 shoots the photographic object to obtain multiple pictures or multiple videos, including:

Shooting the subject according to the shooting parameters to obtain multiple pictures or multiple videos.

In an optional implementation manner, the control module 302 photographs the photographed object through a photographing device to obtain multiple pictures or multiple videos, including:

Adjusting the shooting parameters by a preset amplitude;

Based on the adjusted shooting parameters, shooting the subject to obtain multiple pictures or multiple videos.

Please refer to FIG. 5, which is a schematic structural diagram of a drone provided by an embodiment of the present invention. Among them, the drone shown in FIG. 5 may also include each module shown in FIG. 4 to perform operations of each module shown in FIG. 4. As shown in FIG. 5, the drone may include at least one controller 401, such as a CPU; at least one memory 402, and a communication device 403. In addition, the drone also includes at least one photographing device 404. The photographing device 404 can be directly mounted on the body or the main body of the drone, or can be installed on the gimbal, and mounted on the drone through the gimbal. The controller 401, the memory 402, the communication device 403, and the imaging device 404 are connected via a bus 405.

The memory 402 is configured to store program instructions;

The controller 401 is configured to execute a program instruction stored in the memory 402. When the program instruction is executed, the controller 401 is configured to perform the following steps:

By obtaining the positional relationship between the subject and the drone;

Controlling the drone to maintain the positional relationship with the photographic subject, and photographing the photographic subject through the photographing device 404 to obtain multiple pictures or multiple videos.

The multiple pictures or videos are used to generate a stop motion animation at a preset frame rate. In an optional implementation manner, the controller 401 may generate a stop motion animation at a preset frame rate based on the multiple pictures or videos. In another optional implementation manner, the controller 401 may send the multiple pictures or videos to the display device or a remote control device integrated with the display device through the communication device 403, and the display device or the remote control device may The multiple pictures or videos generate a stop motion animation at a preset frame rate.

In an optional implementation manner, the controller 401 calls a program instruction stored in the memory 402 to obtain a positional relationship between the photographic object and the drone, which is specifically:

Acquiring a shooting picture selected by a user, and acquiring depth information of a shooting scene where the shooting picture is located based on a visual sensor;

Based on the shooting picture and the depth information, a positional relationship between the drone and a shooting object in the shooting picture is obtained.

In an optional implementation manner, the control rule is specifically configured to:

Controlling the drone to maintain the positional relationship with the photographed subject while the drone is in flight;

Alternatively, when the drone is in a hovering state, control the drone to maintain the positional relationship with the photographic subject.

In an optional implementation manner, the controller 401 invokes a program instruction stored in the memory 402, and uses the photographing device to photograph the photographed object according to the photographing parameters to obtain multiple pictures, specifically:

Receiving shooting instructions input by the user;

In response to the shooting instruction, shooting the subject through the shooting device to obtain multiple pictures.

In an optional implementation manner, the controller 401 calls a program instruction stored in the memory 402 to shoot the photographed object to obtain multiple pictures or multiple videos, specifically:

In an optional implementation manner, the controller 401 calls program instructions stored in the memory 402 to control the drone flying based on a control rule, specifically:

Receiving a follow mode command;

Correspondingly, the controller calls a program instruction stored in the memory 402 to shoot the subject to obtain multiple pictures or videos. Specifically, the controller shoots the subject according to the shooting parameters to obtain multiple pictures. Pictures or multiple videos.

Adjusting the shooting parameters by a preset amplitude;

For specific implementation of the controller 401 in this embodiment of the present invention, reference may be made to the description of related content in the foregoing embodiments, and details are not described herein.

An embodiment of the present invention further provides a drone system. The drone system includes a control device, a photographing device, and a drone. The photographing device can be set on a gimbal and then mounted on the drone through the gimbal. Or the camera can be mounted directly on the drone.

The drone is used to obtain the positional relationship between the subject and the drone, and controls the drone to fly based on a control rule, and the control rule is used to control the drone to maintain a relationship with the subject. The positional relationship between them; shooting the subject to obtain multiple pictures or videos; wherein the multiple pictures or videos are used to generate a stop motion animation at a preset frame rate.

The stop motion animation may be generated by a drone, or the drone may send the multiple pictures or videos to a control device, and the control device generates a stop motion animation with a preset frame rate.

In an optional implementation manner, the drone is used to obtain the positional relationship between the shooting object and the drone, which is specifically:

Acquiring a shooting picture selected by a user, and acquiring depth information of the shooting object with respect to a shooting device or a drone based on a visual sensor;

Based on the depth information, a positional relationship between the drone and the subject is obtained.

During the flight of the drone, the drone controls the drone to maintain the position relationship with the photographic subject;

Alternatively, when the drone is in a hovering state, the drone controls the drone to maintain the positional relationship with the photographic subject.

In an optional implementation manner, the drone is configured to capture the photographic object through a photographing device to obtain multiple pictures or multiple videos, specifically:

The drone receives the shooting instruction input by the user;

In response to the shooting instruction, the drone shoots the subject through the shooting device to obtain multiple pictures or multiple videos.

At a preset time interval, the drone shoots the subject through the shooting device to obtain multiple pictures or multiple videos.

In an optional implementation manner, the drone is used to control the drone's flight based on the control rules, specifically:

Receiving a follow mode command;

In another optional embodiment, the drone is used to control the drone's flight based on the control rules. Specifically, if the following mode command received is a non-following mode, the drone cannot intelligently adjust the drone. At least one of the posture of the camera, the posture of the photographing device, and the posture of the gimbal where the photographing device is located to follow the photographic subject. Instead, the control device adjusts at least one of the posture of the drone, the posture of the photographing device, and the posture of the pan / tilt where the photographing device is located by the user's operation to follow the photographic subject.

Correspondingly, the drone photographs the subject through the shooting device to obtain multiple pictures or videos, including:

Adjusting the shooting parameters by a preset amplitude;

A computer-readable storage medium is also provided in the embodiment of the present invention. The computer-readable storage medium stores a computer program, and the computer program is executed by a controller to implement the description in the embodiment corresponding to FIG. 3 of the present invention. The image processing method can also implement the device corresponding to the embodiment of the present invention shown in FIG. 4 or FIG. 5, which is not described again here.

A computer program product is also provided in the embodiment of the present invention. The computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to implement the present invention. FIG. 3 The image processing method described in the corresponding embodiment can also implement the device of the corresponding embodiment of the present invention described in FIG. 4 or FIG. 5, and details are not described herein again.

The computer-readable storage medium may be an internal storage unit of the device according to any one of the foregoing embodiments, such as a hard disk or a memory of the device. The computer-readable storage medium may also be an external storage device of the device, such as a plug-in hard disk, a Smart Media Card (SMC), and a Secure Digital (SD) card provided on the device. , Flash card (Flash card) and so on. Further, the computer-readable storage medium may also include both an internal storage unit of the device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal. The computer-readable storage medium may also be used to temporarily store data that has been or will be output.

A person of ordinary skill in the art can understand that all or part of the processes in the methods of the foregoing embodiments can be implemented by using a computer program to instruct related hardware. The program can be stored in a computer-readable storage medium. The program When executed, the processes of the embodiments of the methods described above may be included. The storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random, Access Memory, RAM).

The above disclosure is only the preferred embodiments of the present invention, and of course, the scope of the rights of the present invention cannot be limited by this. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope of the present invention.

Claims

An image processing method, which is characterized in that the method is applied to a drone including a photographing device, and the method includes:

Obtain the positional relationship between the subject and the drone;

Controlling the drone flying based on a control rule, the control rule is used to control the drone to maintain the position relationship with the photographic subject;

A plurality of image data are obtained by photographing the photographed subject through the photographing device, and the plurality of image data is used to generate a stop motion animation at a preset frame rate.
The method according to claim 1, wherein the acquiring a position relationship between a photographic object and the drone comprises:

Acquiring a shooting picture selected by a user, and acquiring depth information of a shooting object in the shooting picture relative to the shooting device or the drone based on a visual sensor;

Based on the depth information, a positional relationship between the drone and the subject is obtained.
The method according to claim 1, wherein the control rule is specifically configured to:

Controlling the drone to maintain the positional relationship with the photographed subject while the drone is in flight;

Alternatively, when the drone is in a hovering state, control the drone to maintain the positional relationship with the photographic subject.
The method according to claim 3, wherein the obtaining the plurality of image data by photographing the photographed subject through the photographing device comprises:

Receiving shooting instructions input by the user;

In response to the shooting instruction, shooting the subject through the shooting device to obtain a plurality of image data.
The method according to claim 3, wherein the obtaining the plurality of image data by photographing the photographed subject through the photographing device comprises:

At a preset time interval, the photographing subject is photographed by the photographing device to obtain a plurality of image data.
The method according to claim 3, wherein the controlling the drone flight based on a control rule comprises:

Receiving a follow mode command;

When the following mode instruction is a following mode, adjust at least one of a posture of the drone, a posture of the photographing device, and a posture of a pan / tilt where the photographing device is located to follow the photographed subject. .
The method according to claim 1, wherein the acquiring a position relationship between a photographic object and the drone comprises:

Obtaining the positional relationship and shooting parameters between the shooting object and the drone from a database, and the database stores in advance the positional relationship and the shooting parameters required for shooting a stop motion animation;

Obtaining the plurality of image data by photographing the photographic subject through a photographing device includes:

Photographing the photographed subject according to the photographing parameters to obtain a plurality of image data.
The method according to claim 1, wherein the acquiring a plurality of image data by photographing the photographed subject by using a photographing device comprises:

Adjust the shooting parameters of each image data by preset amplitude;

Based on the adjusted shooting parameters, the shooting device shoots the shooting object to obtain multiple image data.
An image processing device, characterized in that the image processing device is applied to a drone system including a photographing device, and the image processing device includes a memory, a controller, and a photographing device;

The memory is used to store program instructions;

The controller executes program instructions stored in the memory. When the program instructions are executed, the controller is configured to perform the following steps:

Obtain the positional relationship between the subject and the drone;

Controlling the drone flying based on a control rule, the control rule is used to control the drone to maintain the position relationship with the photographic subject;

A plurality of image data are obtained by photographing the photographed subject through the photographing device, and the plurality of image data is used to generate a stop motion animation at a preset frame rate.
The device according to claim 9, wherein the acquiring the positional relationship between the photographic subject and the drone is specifically:

Acquiring a shooting picture selected by a user, and acquiring depth information of a shooting object with respect to the shooting device or the drone based on a visual sensor;

Based on the depth information, a positional relationship between the drone and the subject is obtained.
The device according to claim 9, wherein the control rule is specifically configured to:

Controlling the drone to maintain the positional relationship with the photographed subject while the drone is in flight;

Alternatively, when the drone is in a hovering state, control the drone to maintain the positional relationship with the photographic subject.
The device according to claim 11, wherein the controller obtains a plurality of image data by shooting the photographed subject through the photographing device, specifically:

Receiving shooting instructions input by the user;

In response to the shooting instruction, shooting the subject through the shooting device to obtain a plurality of image data.
The device according to claim 11, wherein the controller obtains a plurality of image data by shooting the photographed subject through the photographing device, specifically:

At a preset time interval, the photographing subject is photographed by the photographing device to obtain a plurality of image data.
The device according to claim 11, wherein the controller controls the drone flight based on a control rule, specifically:

Receiving a follow mode command;

When the following mode instruction is a following mode, adjust at least one of a posture of the drone, a posture of the photographing device, and a posture of a pan / tilt where the photographing device is located to follow the photographic subject. .
The device according to claim 9, wherein the acquiring the positional relationship between the photographic subject and the drone is specifically:

Obtaining the positional relationship and shooting parameters between the shooting object and the drone from a database, and the database stores in advance the positional relationship and the shooting parameters required for shooting a stop motion animation;

The controller photographs the photographed object to obtain multiple image data, specifically:

Photographing the photographed subject according to the photographing parameters to obtain a plurality of image data.
The device according to claim 9, wherein the controller obtains a plurality of image data by shooting the photographed object through the photographing device, specifically:

Adjusting the shooting parameters for shooting each image data with a preset amplitude;

A plurality of image data are obtained by using the photographing device to photograph the photographic subject based on the adjusted photographing parameters.
An unmanned aerial vehicle, which is characterized by comprising a body and a shooting device of the unmanned aerial vehicle;

The photographing device is mounted on the body of the drone, or is provided on a gimbal, and is mounted on the body of the drone through the gimbal;

The drone further includes a controller and a memory, and the controller is configured to call code stored in the memory to execute the image processing method according to any one of claims 1 to 8.
A computer-readable storage medium, characterized in that the computer storage medium stores a computer program, the computer program includes program instructions, and the program instructions, when executed by a controller, cause the controller to execute the method according to claim 1 The image processing method according to any one of to 8.
A computer program product, characterized in that the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to implement any one of claims 1 to 8 The image processing method described above.