WO2021134183A1

WO2021134183A1 - Image processing method and device, mobile platform, camera device, and storage medium

Info

Publication number: WO2021134183A1
Application number: PCT/CN2019/129860
Authority: WO
Inventors: 屈信超; 李琛
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-12-30
Filing date: 2019-12-30
Publication date: 2021-07-08
Also published as: CN112616325A

Abstract

An image processing method and device, a mobile platform, a camera device, and a storage medium, which can solve the problem of a scene changing too fast. The image processing method comprises: acquiring a first frame and a second frame, a viewing angle direction of the first frame being different from that of the second frame (S100); and determining a target image according to the first frame and the second frame, the first frame and the second frame both containing a scene in the target image (S200).

Description

Image processing method, device, movable platform, camera device and storage medium

Technical field

The present invention relates to the field of image technology, in particular to an image processing method, device, movable platform, camera device and storage medium.

Background technique

During image playback, scenes of different images may change too quickly. In this case, the overall playback screen will be blurred and affect the user's viewing experience. The scene change of the image too fast may be because different imaging devices collect images from different viewing angles; it may also be because the viewing angle changes too fast when the imaging device changes different viewing angles to collect images.

For example, in the process of aerial photography, drones often need to rotate to change the direction of view. If the rotation speed of the drone is large, the direction of the viewing angle will change quickly when images are collected. Then, subsequent images may appear when the image is played. The scene in the playback screen changes quickly.

Summary of the invention

The invention provides an image processing method, a device, a movable platform, a camera device and a storage medium, which can solve the problem of excessively fast scene changes.

In a first aspect of the embodiments of the present invention, an image processing method is provided, including:

Acquiring a first image frame and a second image frame, where the viewing angle directions of the first image frame and the second image frame are different;

The target image is determined according to the first image frame and the second image frame, and both the first image frame and the second image frame include a scene in the target image.

In a second aspect of the embodiments of the present invention, an image processing device is provided, including a memory and a processor;

The memory is used to store program code;

The processor is used to call the program code, and when the program code is executed, it is used to implement the image processing method described in the foregoing embodiment.

In a third aspect of the embodiments of the present invention, a movable platform is provided, including:

Body

A power system, arranged in the body, and the power system is used to provide power for the movable platform;

An image acquisition device, which is provided in the body, and is used to acquire image frames;

Memory for storing program codes; and

In a fourth aspect of the embodiments of the present invention, a camera device is provided, including:

shell;

The lens assembly is arranged inside the housing;

Memory for storing program codes; and

In a fifth aspect of the embodiments of the present invention, a computer-readable storage medium is provided,

The computer-readable storage medium stores program code, and when the program code is executed, the image processing method as described in the foregoing embodiment is implemented.

Based on the above technical solution, in the embodiment of the present invention, when the first image frame and the second image frame with different viewing angle directions are acquired, the target image can be determined according to the two image frames, and the first image frame and the second image frame are both The scene in the target image is included. In other words, the scene in the target image is included in the first image frame and the second image frame. Then, when the target image is inserted between the first image frame and the second image frame for playback, Or, inserting the target image into the first image frame and the second image frame to play between the processed images, can play the role of scene transition, slow down the speed of scene changes during playback, and solve the problem of excessive scene changes .

Description of the drawings

In order to explain the technical solutions in the embodiments of the present invention more clearly, the following will briefly introduce the drawings that need to be used in the embodiments of the present invention. Obviously, the drawings in the following description are only some of the implementations recorded in the present invention. For example, for those of ordinary skill in the art, other drawings can be obtained based on these drawings of the embodiments of the present invention.

FIG. 1 is a schematic flowchart of an image processing method according to an embodiment of the present invention;

2 is a schematic diagram of the relationship between the first plane and the second plane according to an embodiment of the present invention;

3 is a schematic diagram of the relationship between the first plane, the second plane and the target plane according to an embodiment of the present invention;

Fig. 4 is a structural block diagram of an image processing apparatus according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention. In addition, if there is no conflict, the following embodiments and features in the embodiments can be combined with each other.

The terms used in the present invention are only for the purpose of describing specific embodiments, rather than limiting the present invention. The singular forms of "a", "said" and "the" used in the present invention and claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should be understood that the term "and/or" as used herein refers to any or all possible combinations of one or more associated listed items.

Although the terms first, second, third, etc. may be used in the present invention to describe various information, the information should not be limited to these terms. These terms are used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information. Depending on the context, in addition, the word "if" used can be interpreted as "when", or "when", or "in response to certainty."

The image processing method of the embodiment of the present invention will be described in detail below, but this should not be used as a limitation.

In an embodiment, referring to FIG. 1, an image processing method may include the following steps:

S100: Acquire a first image frame and a second image frame, where the viewing angle directions of the first image frame and the second image frame are different;

S200: Determine a target image according to the first image frame and the second image frame, where both the first image frame and the second image frame include a scene in the target image.

In the embodiment of the present invention, the execution subject of the image processing method may be an electronic device. The electronic device may be an imaging device, for example, a camera device, or a movable film platform equipped with a camera device, such as a drone, a vehicle, and the like. Of course, the electronic device can also be other devices, as long as it has a certain image processing capability.

In the embodiment of the present invention, the viewing angle direction of the image frame may refer to the direction from the bisector of the lens viewing angle to the captured scene when the image frame is captured.

In step S100, a first image frame and a second image frame are acquired.

The first image frame and the second image frame may be acquired by the same imaging device, or may be acquired by different imaging devices. The first image frame and the second image frame may be collected by an external imaging device; or, in the case where the executing main electronic device is an imaging device, the first image frame and the second image frame may be collected by the electronic device itself.

The viewing angles of the first image frame and the second image frame are different. In other words, the viewing angles of the first image frame and the second image frame are different by a certain angle. Therefore, the scenes in the first image frame and the second image frame will also be different. different.

The first image frame and the second image frame may be two image frames collected at the same location but different viewing angle directions, or may also be two image frames collected at different locations and different viewing angle directions.

Since the viewing angles of the first image frame and the second image frame are different, if the first image frame and the second image frame are played continuously and the playing speed is fast (the time interval between the two frames is short), the scene change will occur Faster question. The embodiment of the present invention solves this problem through subsequent steps.

To solve this problem, in the embodiment of the present invention, after acquiring the first image frame and the second image frame, step S200 is further executed: determining the target image according to the first image frame and the second image frame.

Both the first image frame and the second image frame include a scene in the target image. In other words, some scenes in the first image frame and the second image frame are the same, and the scene in the target image is the partial scene or a part of the partial scene.

The angle between the viewing angles of the first image frame and the second image frame needs to ensure that there is an intersection of the field of view when the first image frame and the second image frame are collected. In this way, the first image frame and the second image There are some scenes in the frame that are the same.

In order to ensure that when the angle between the viewing angles of the first image frame and the second image frame is large, there is still a large intersection in the field of view when the first image frame and the second image frame are collected. The FOV (field of view, angle of view) lens is used to collect the first image frame and the second image frame.

When determining the target image according to the first image frame and the second image frame, the same scene in the first image frame and the second image frame (hereinafter referred to as the first scene, the first scene is a part of the first image frame) Scene, which is also a part of the scene in the second image frame) to determine the target image. Optionally, the first image area where the first scene is located in the first image frame and the second image area where the first scene is located in the second image frame may be determined, based on the first image area and the second image area To generate the target image, for example, the two image regions may be subjected to affine transformation first, and then fusion processing is performed after the affine transformation to generate the target image.

It can be understood that the foregoing is only an example of determining the target image and is not meant as a limitation. The target image can also be determined in other ways, as long as it is ensured that the scene in the target image exists in the first image frame and the second image frame.

The determined target image may be one or more. In the case of multiple, the field of view corresponding to different target images may be different. Correspondingly, the scenes in different target images may be different, but they all exist in the first image frame and the second image frame.

In the embodiment of the present invention, when the first image frame and the second image frame with different viewing angle directions are acquired, the target image can be determined according to the two image frames, and the first image frame and the second image frame both contain the target image. In other words, the scene in the target image is contained in the first image frame and the second image frame. Then, when the target image is inserted between the first image frame and the second image frame, or the target image When inserted between the processed images of the first image frame and the second image frame, it can play the role of scene transition, slow down the speed of scene changes during playback, and solve the problem of excessive scene changes.

In the drone's aerial scene, for example, when the angle of view needs to be changed by rotating or tilting, the speed of the screen playback can be controlled by the above methods, so as to avoid excessive changes in the speed of the drone, which may cause the video to be blurred and sometimes clear. The phenomenon.

In one embodiment, in step S200, determining the target image according to the first image frame and the second image frame includes:

The first image frame and the second image frame are projected onto the same target plane to generate the target image; wherein the viewing angle direction corresponding to the target plane is different from the viewing angle direction of the first image frame and the second image frame .

Since the perspective directions of the first image frame and the second image frame are different, the same scene (ie, the first scene) in the first image frame and the second image frame is presented in the first image frame and the second image frame at different angles Wherein, projecting the first image frame and the second image frame onto the same target plane can make the angles of the scenes (including the first scene) in the two image frames obtained by the projection consistent, so as to generate the desired target image.

The viewing angle direction corresponding to the target plane is different from the viewing angle directions of the first image frame and the second image frame, which can ensure that when the target image is inserted between the first image frame and the second image frame, or the target image is inserted When the first image frame and the second image frame are played between the processed images, while slowing down the scene change, the angle of the scene can be gradually adjusted to make the transition of the scene change smoother and avoid sudden scene changes.

To project the first image frame and the second image frame onto the same target plane, either the first image frame and the second image frame can be directly projected onto the target plane respectively, or the first image frame and the second image frame can be respectively projected onto the target plane. The frame is projected onto the curved surface, and then projected from the curved surface onto the target plane, and the specific method is not limited.

In one embodiment, in step S200, projecting the first image frame and the second image frame onto the same target plane to generate the target image includes:

The first image frame and the second image frame are projected onto the same target plane and then subjected to fusion processing to generate the target image.

After the first image frame and the second image frame are projected onto the same target plane, the two obtained image frames can be fused to obtain a target image that meets the requirements.

Of course, in addition to fusion processing after projection, other image processing methods, such as image interception, can also be included, which are not specifically limited.

In one embodiment, in step S200, the first image frame and the second image frame are projected onto the same target plane and then merged to generate the target image, which may include the following steps:

S201: Acquire a projection relationship corresponding to the target plane, where the projection relationship includes a first projection relationship for projecting a first image frame and a second projection relationship for projecting a second image frame;

S202: Project the first image frame from the first plane where it is located to the target plane according to the first projection relationship to obtain a third image frame;

S203: Project the second image frame from the second plane where it is located to the target plane according to the second projection relationship to obtain a fourth image frame;

S204: Perform fusion processing on the third image frame and the fourth image frame to obtain a target image in the target plane;

Wherein, the first plane, the second plane and the target plane are imaging planes in different viewing angle directions.

The first projection relationship may be determined according to the relationship between the first plane and the target plane, or the relationship between the viewing angle direction corresponding to the first plane and the viewing angle direction corresponding to the target plane, and may be used to convert the image frame on the first plane Project to the target plane.

According to the first projection relationship, the first image frame is projected from the first plane where it is located to the target plane to obtain the third image frame. Compared with the first image frame, the third image frame has the same scene, but the angle of the scene in the image frame is different.

The second projection relationship can be determined according to the relationship between the second plane and the target plane, or the relationship between the viewing angle direction corresponding to the second plane and the viewing angle direction corresponding to the target plane, and can be used to convert the image on the second plane The frame is projected onto the target plane.

The second image frame is projected from the second plane where it is located to the target plane according to the second projection relationship to obtain the fourth image frame. Compared with the second image frame, the fourth image frame has the same scene, but the angle of the scene in the image frame is different.

After the projection is completed, the third image frame and the fourth image frame are fused to obtain the target image in the target plane.

In one embodiment, the first image frame and the second image frame are acquired by the same imaging device during rotation, and the first plane is the imaging plane of the imaging device at a first rotation angle The second plane is the imaging plane of the imaging device at a second rotation angle, and the target plane corresponds to the imaging plane of the imaging device at a third rotation angle.

The first image frame and the second image frame are collected by the same imaging device during the rotation process, so the viewing angle direction at the time of collection changes with the rotation angle of the imaging device. The imaging device collects an image at the first rotation angle to obtain a first image frame imaged on the first plane. The imaging device collects an image at a second rotation angle to obtain a second image frame imaged on the second plane. The first rotation angle and the second rotation angle are different, and correspondingly, the viewing angle directions of the first image frame and the second image frame are also different.

The target image is not a captured image, but an image generated based on the first and second captured image frames. However, the target image is an image in the target plane, which corresponds to the imaging device at the third rotation angle Therefore, assuming the image frame acquired by the imaging device at the third rotation angle, at least part of the scene in the image frame is the same as the scene in the target image, and the angle of the scene is also the same.

Preferably, the third rotation angle is between the first rotation angle and the second rotation angle. The target image may serve as an image frame or a partial image area in the image frame acquired by the imaging device after the first image frame and before the second image frame during the rotation process.

In this way, it can be realized that when the target image is inserted into the first image frame and the second image frame for playback, or the target image is inserted into the first image frame and the second image frame to be played between the processed images, While slowing down changes in the scene, the angle of the scene can also be gradually changed.

In one embodiment, the first projection relationship is determined based on the angle difference between the first rotation angle and the third rotation angle, and the second projection relationship is determined based on the second rotation angle and the The angle difference between the third rotation angles is determined.

The first rotation angle and the second rotation angle may be obtained from an imaging device. For example, an inertial sensor is provided in the imaging device, and the first rotation angle and the second rotation angle may be obtained from the inertial sensor.

The angle difference between the first rotation angle and the third rotation angle may be preset. In the case where the angle difference and the first rotation angle are determined, the third rotation angle can be calculated, and then the second rotation angle can be calculated The angle difference from the third angle of rotation. Of course, the determination method of the angle difference between the first rotation angle and the third rotation angle, and the angle difference between the second rotation angle and the third rotation angle is not limited to this.

When the angle difference between the first rotation angle and the third rotation angle is determined, the projection transformation matrix from the first plane corresponding to the first rotation angle to the target plane corresponding to the third rotation angle can be calculated, and the projection The transformation matrix is determined as the first projection relationship.

When the angle difference between the second rotation angle and the third rotation angle is determined, the projection transformation matrix from the second plane corresponding to the second rotation angle to the target plane corresponding to the third rotation angle can be calculated, and the projection The transformation matrix is determined as the second projection relationship.

In an embodiment, the first image frame and the second image frame may also be acquired by different imaging devices with different viewing angle directions.

The two imaging devices can collect the first image frame and the second image frame in different viewing angle directions when they are at the same position or when the positions are not far apart.

The method of determining the target image in the foregoing embodiment is still applicable to this situation, but when determining the first projection relationship and the second projection relationship, the rotation angle of the two imaging devices is considered. Of course, if necessary, The positional relationship between two imaging devices can also be considered.

In an embodiment, in step S204, the fusion processing of the third image frame and the fourth image frame to generate a target image in the target plane may include the following steps:

S2041: Cut out a first image from the third image frame, and cut out a second image from the fourth image frame, where the first image is the same as the scene in the second image;

S2042: Fuse the first image and the second image to obtain the target image.

Part of the scene (ie, the first scene) in the first image frame and the second image frame is the same, so the first scene in the third image frame and the fourth image frame projected to the target plane is still the same, but this The angle of the first scene presented in the image frame has undergone some changes.

When the first image is intercepted from the third image frame, the image area where the first scene is located in the third image frame may be intercepted, or a part of the area from the image area may be intercepted as the first image. When the second image is cut out from the fourth image frame, the image area where the first scene is located in the fourth image frame may be cut out, or a part of the area in the image area may be cut out as the second image.

Of course, the specific interception method and the interception size are not limited, as long as the scenes in the first image and the second image are the same.

After the first image and the second image are obtained, the first image and the second image can be fused according to a preset image fusion algorithm to obtain the target image. The image fusion algorithm may include, for example, a pyramid fusion algorithm, of course, it may also be other image fusion algorithms, and the details are not limited.

In one embodiment, the position of the first image on the target plane is the same as the position of the second image on the target plane.

The third image frame and the fourth image frame have an intersection area in the target plane, and the scenes of the third image frame and the fourth image frame in the intersection area are the same. Therefore, the image area at the same position can be respectively intercepted from the intersection area of the third image frame and the fourth image frame as the first image and the second image, ensuring the position of the first image on the target plane and the position of the second image on the target plane. Same, then the scenes in the first image and the second image are also the same.

The intersection area is generally at the center position of the field of view corresponding to the target plane in the target plane. The position of the target area required for interception can be determined in advance according to the center position of the field of view, and the third image frame and the first image frame in the target plane are obtained by projection. In the case of four image frames, the area corresponding to the position of the target area can be intercepted from the third image frame and the fourth image frame, respectively, as the first image and the second image.

In the following, with reference to FIG. 2 and FIG. 3, an example is given to illustrate the method of generating a target image in an embodiment of the present invention.

The plane where a1b1 is located is the imaging plane when the imaging device collects the first image frame, that is, the first plane, ∠a1Ob1 is the angle of view corresponding to the first plane, the direction of the bisector of ∠a1Ob1 to the scene is the direction of the angle of view corresponding to the first plane, and a1b1 is The field of view corresponding to the first plane.

The plane where a2b2 is located is the imaging plane when the imaging device collects the second image frame, that is, the second plane, ∠ a2Ob2 is the viewing angle corresponding to the second plane, the direction of the bisector of ∠a2Ob2 to the scene is the viewing angle direction corresponding to the second plane, a2b2 is The field of view corresponding to the first plane.

The plane where a3b3 is located is the target plane, ∠a3Ob3 is the view angle corresponding to the target plane, the direction of the bisector of ∠a3Ob3 to the scene is the view angle direction corresponding to the target plane, and a3b3 is the field of view corresponding to the target plane.

The viewing angle direction corresponding to the plane where a1b1 and a2b2 are located is different, and the angle difference is β; the viewing angle direction corresponding to the plane where a3b3 is located is different from the viewing angle direction corresponding to the plane where a1b1 is located and the plane where a2b2 is located. The angle difference between the viewing angle directions corresponding to the plane where a1b1 is located is α, and correspondingly, the angle difference between the viewing angle direction corresponding to the plane where a3b3 is located and the viewing angle direction corresponding to the plane where a2b2 is located is β-α.

According to α, the first projection relationship between the plane of a1b1 and the plane of a3b3 can be determined, and the second projection relationship between the plane of a2b2 and the plane of a3b3 can be determined according to β-α.

According to the first projection relationship, project the first image frame from the plane of a1b1 to the plane of a3b3 to obtain the third image frame; according to the second projection relationship, project the second image frame from the plane of a2b2 to the plane of a3b3 to obtain the fourth image frame.

Then, the area corresponding to the preset target area position is cut out from the third image frame as the first image, and the area corresponding to the preset target area position is cut out from the fourth image frame as the second image, where, The target area position is an area position centered on the target plane, that is, the center position of the field of view a3b3 corresponding to the plane where a3b3 is located.

Then, a preset fusion algorithm is used to fuse the first image and the second image to obtain the target image.

In the above example, after the imaging device collects the first image frame, it rotates by a certain angle β, and then collects the second image frame. When the rotation speed is faster, the β is larger, and the scene changes in the first image frame and the second image frame are also greater, but the two fields of view have an intersection, so there are parts in the first image frame and the second image frame The scene (that is, the first scene) is the same. In the embodiment of the present invention, a target image including the first scene or a part of the first scene is generated according to the first image frame and the second image frame, and the scene in the target image can be used as the first image frame and the second image frame The transition of the middle scene, slow down the scene change between images.

In an embodiment, after the target image is determined according to the first image frame and the second image frame, the method further includes the following steps:

S300: Determine an image sequence in which a third image and a fourth image are located, the third image is an image intercepted from the first image frame, and the fourth image is an image intercepted from the second image frame ；

S400: Insert the target image between the third image and the fourth image of the image sequence to generate a video stream for playback.

When the third image is cut out from the first image frame, the central area of the first image frame may be cut out as the third image. When the fourth image is cut out from the second image frame, the central area of the second image frame may be cut out as the fourth image. Of course, the specific intercepting position is not limited, and it is only a preferred way here.

The images in the image sequence where the third image and the fourth image are located are used for playback, and the size of all images in the image sequence may be the same, and the specific size may be determined according to playback requirements.

The target image is inserted into the third image and the fourth image of the image sequence, and the obtained image sequence is played as a video stream. The size of the target image can be the same as the size of each image in the image sequence.

In this way, when the video stream is played, the third image, target image, and fourth image are played in sequence, or the fourth image, target image, and third image are played in sequence. The scenes in the target image are used as the third image and the fourth image. The transition of scenes can make the video stream play more smoothly, and avoid the problem of too fast scene switching between video frames, which causes the naked eye to look blurry.

Of course, in the video stream being played, there may also be other images inserted in the above-mentioned manner in the embodiment of the present invention.

For example, half of the images in the video stream are intercepted from the captured image frames, and the other half of the images are inserted through the above method, and every two images intercepted from the captured image frames are inserted through the above method Determine the target image. In this case, assuming that the video stream is played at the playback frame rate f1, the scene change speed when playing the video stream is only 1/2 of the scene change speed of the captured image frame, which greatly slows down the scene. Speed of change.

In one embodiment, the first image frame and the second image frame are two image frames collected by the imaging device during rotation.

Preferably, the first image frame and the second image frame are two adjacent image frames collected by the imaging device during rotation. The intersection range of the fields of view of adjacent image frames is larger, which is more conducive to obtaining the target image.

Of course, the first image frame and the second image frame may also be two frames of images separated by one or several frames, as long as it is ensured that the fields of view of the first image frame and the second image frame overlap.

In an embodiment, in step S100, obtaining the first image frame and the second image frame may include the following:

When it is determined that the imaging device is rotating, obtain the current image frame collected by the imaging device and obtain the previous historical image frame of the current image frame, determine the current image frame as the second image frame and determine the historical image frame as the The first image frame.

The rotation of the imaging device is usually triggered by an instruction. When triggered by the instruction, it can be determined that the imaging device is rotating. Of course, the specific method for determining the rotation of the imaging device is not limited to this.

The first image frame collected by the imaging device can be buffered first, and then when collecting, the current image frame currently collected can be used as the second image frame, and the previous historical image frame of the current image frame can be used as the first image frame.

In this way, the imaging device can perform the above steps of determining and inserting the target image every time an image frame is collected, thereby slowing down the scene change speed of the entire video stream.

In an embodiment, after acquiring the current image frame collected by the imaging device, the method further includes:

Performing distortion correction on the acquired current image frame to correct scene distortion generated in the current image frame;

The determining the current image frame as the second image frame includes:

The current image frame after the distortion correction is determined as the second image frame.

Due to the hardware of the imaging device, etc., the captured image frame may be deformed, such as stretching at the edge of the image frame. In order to avoid the obtained target image from being affected by the distortion, the current image frame is corrected for distortion. Correct the distortion of the scene in the current image frame.

When performing distortion correction on the current image frame, the distortion correction can be realized according to the principle of pinhole imaging and the camera parameters of the imaging device. The specific distortion correction method is not limited, as long as the scene distortion generated in the current image frame can be corrected.

Optionally, the acquired historical image frames may be image frames that have undergone distortion correction. If the historical image frame has not undergone distortion correction, then the historical image frame may also be subjected to distortion correction to correct the scene deformation generated in the historical image frame.

After performing distortion correction on the current image frame, the current image frame can also be buffered and used as the previous historical image frame of the image frame when the next image frame is collected.

Optionally, if the imaging device that collects the first image frame and the second image frame has jitter, electronic image stabilization, optical image stabilization, and other technologies can be used to achieve image stabilization to reduce the first plane and target plane, and the second plane and target. The error between the planes.

Based on the same inventive concept as the above method, the present invention also provides an image processing device. Referring to FIG. 4, the image processing device 100 may include a processor 102 and a memory 101.

In one embodiment, continuing to refer to FIG. 4, the memory 101 is used to store program codes;

The processor 102 is configured to call the program code, and when the program code is executed, it is configured to implement the image processing method described in the foregoing embodiment.

Based on the same inventive concept as the above method, the present invention also provides a movable platform, which may include:

Body

Memory for storing program codes; and

In one embodiment, the movable platform is a drone.

Based on the same inventive concept as the above method, the present invention also provides an imaging device, which may include:

shell;

The lens assembly is arranged inside the housing;

Memory for storing program codes; and

Based on the same inventive concept as the above method, an embodiment of the present invention also provides a computer-readable storage medium with program code stored on it, and when the program code is executed (for example, the program code is executed by a processor), it can be realized The image processing method described in Figure 1.

The systems, devices, modules, or units explained in the above embodiments may be implemented by computer chips or entities, or implemented by products with certain functions. A typical implementation device is a computer. The specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email receiving and sending device, and a game control A console, a tablet computer, a wearable device, or a combination of any of these devices.

For the convenience of description, when describing the above device, the functions are divided into various units and described separately. Of course, when implementing the present invention, the functions of each unit can be implemented in the same one or more software and/or hardware.

Those skilled in the art should understand that the embodiments of the present invention may be provided as a method, a system, or a computer program product. Therefore, the present invention may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may adopt the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

The present invention is described with reference to flowcharts and/or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and/or block in the flowchart and/or block diagram, and the combination of processes and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing equipment are generated for use. It is a device that realizes the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

Moreover, these computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device, The instruction device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be loaded into a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, thereby executing instructions on the computer or other programmable equipment Provides steps for realizing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

The foregoing descriptions are merely embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention can have various modifications and changes. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

An image processing method, characterized in that it comprises:

Acquiring a first image frame and a second image frame, where the viewing angle directions of the first image frame and the second image frame are different;

The target image is determined according to the first image frame and the second image frame, and both the first image frame and the second image frame include a scene in the target image.
8. The image processing method of claim 1, wherein determining the target image according to the first image frame and the second image frame comprises:

The first image frame and the second image frame are projected onto the same target plane to generate the target image; wherein the viewing angle direction corresponding to the target plane is different from the viewing angle direction of the first image frame and the second image frame .
3. The image processing method of claim 2, wherein projecting the first image frame and the second image frame onto the same target plane to generate the target image comprises:

The first image frame and the second image frame are projected onto the same target plane and then subjected to fusion processing to generate the target image.
8. The image processing method of claim 3, wherein the first image frame and the second image frame are projected onto the same target plane and then fused to generate the target image, comprising:

Acquiring a projection relationship corresponding to the target plane, the projection relationship including a first projection relationship for projecting a first image frame and a second projection relationship for projecting a second image frame;

Project the first image frame from the first plane where it is located to the target plane according to the first projection relationship to obtain a third image frame;

Project the second image frame from the second plane where it is located to the target plane according to the second projection relationship to obtain a fourth image frame;

Performing fusion processing on the third image frame and the fourth image frame to obtain a target image in the target plane;

Wherein, the first plane, the second plane and the target plane are imaging planes in different viewing angle directions.
The image processing method according to claim 4, wherein:

The first image frame and the second image frame are acquired by the same imaging device during rotation, the first plane is the imaging plane of the imaging device at a first angle of rotation, and the second plane Is the imaging plane of the imaging device at the second rotation angle, and the target plane corresponds to the imaging plane of the imaging device at the third rotation angle.
The image processing method according to claim 5, wherein:

The third rotation angle is between the first rotation angle and the second rotation angle.
5. The image processing method of claim 5, wherein the first projection relationship is determined based on the angle difference between the first rotation angle and the third rotation angle, and the second projection relationship It is determined based on the angle difference between the second rotation angle and the third rotation angle.
The image processing method according to claim 4, wherein:

The first image frame and the second image frame are acquired by different imaging devices with different viewing angle directions.
5. The image processing method according to claim 4, wherein the fusion processing of the third image frame and the fourth image frame to generate a target image in the target plane comprises:

Intercepting a first image from the third image frame, and intercepting a second image from the fourth image frame, where the first image is the same as the scene in the second image;

The first image and the second image are merged to obtain the target image.
9. The image processing method of claim 9, wherein the position of the first image on the target plane is the same as the position of the second image on the target plane.
8. The image processing method of claim 1, wherein after determining the target image according to the first image frame and the second image frame, the method further comprises:

Determining an image sequence in which a third image and a fourth image are located, the third image is an image intercepted from the first image frame, and the fourth image is an image intercepted from the second image frame;

The target image is inserted between the third image and the fourth image of the image sequence to generate a video stream for playback.
11. The image processing method according to any one of claims 1-11, wherein the first image frame and the second image frame are two image frames collected by the imaging device during rotation.
The image processing method according to claim 12, wherein the first image frame and the second image frame are two adjacent image frames collected by the imaging device during the rotation process.
11. The image processing method according to any one of claims 1-11, wherein acquiring the first image frame and the second image frame comprises:

When it is determined that the imaging device is rotating, obtain the current image frame collected by the imaging device and obtain the previous historical image frame of the current image frame, determine the current image frame as the second image frame and determine the historical image frame as the The first image frame.
15. The image processing method of claim 14, wherein after acquiring the current image frame collected by the imaging device, the method further comprises:

Performing distortion correction on the acquired current image frame to correct scene distortion generated in the current image frame;

The determining the current image frame as the second image frame includes:

The current image frame after the distortion correction is determined as the second image frame.
An image processing device, characterized in that it comprises a memory and a processor;

The memory is used to store program code;

The processor is configured to call the program code, and when the program code is executed, it is configured to implement the image processing method according to any one of claims 1-15.
A movable platform, characterized in that it comprises:

Body

A power system, arranged in the body, and the power system is used to provide power for the movable platform;

An image acquisition device, which is provided in the body, and is used to acquire image frames;

Memory for storing program codes; and

The processor is used to call the program code, and when the program code is executed, it is used to implement the image processing method according to any one of claims 1-15.
The movable platform of claim 17, wherein the movable platform is an unmanned aerial vehicle.
A camera device, characterized in that it comprises:

shell;

The lens assembly is arranged inside the housing;

Memory for storing program codes; and

The processor is used to call the program code, and when the program code is executed, it is used to implement the image processing method according to any one of claims 1-15.
A computer-readable storage medium, characterized in that:

The computer-readable storage medium stores program code, and when the program code is executed, the image processing method according to any one of claims 1-15 is realized.