WO2021138768A1

WO2021138768A1 - Method and device for image processing, movable platform, imaging apparatus and storage medium

Info

Publication number: WO2021138768A1
Application number: PCT/CN2020/070487
Authority: WO
Inventors: 李兵; 马骏; 林光远
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-01-06
Filing date: 2020-01-06
Publication date: 2021-07-15

Abstract

A method and device for image processing, a movable platform and a storage medium. The method comprises: acquiring a first video, wherein the first video comprises a plurality of first image frames; selecting at least one target image frame from the plurality of first image frames; acquiring posture data of the first image frame during exposure; and according to the posture data, adjusting the at least one target image frame to obtain a second video. By means of the embodiment, the phenomenon that the second video jitters can be avoided, thus contributing to improving the stability of the video, realizing a smooth video, and improving the viewing experience of a user.

Description

Image processing method, equipment, movable platform, imaging device and storage medium

Technical field

The embodiments of the present invention relate to the field of image processing technology, and in particular to an image processing method, equipment, movable platform, imaging device, and storage medium.

Background technique

Currently, most shooting devices (such as sports cameras) are equipped with an electronic image stabilization (EIS), and the electronic image stabilization algorithm can be used to stabilize the video (or image) acquired by the image acquisition device.

However, for a video (or image) stabilized by the EIS algorithm, if the playback speed of the stabilized video is increased, it may cause the video screen to jitter or twitch, and it is often difficult to obtain a smoothly moving video, which reduces users. Filming and viewing experience.

Summary of the invention

The embodiment of the present invention discloses an image processing method, equipment, a movable platform, an imaging device and a storage medium. Through this implementation manner, the phenomenon of video jitter can be avoided, which is helpful to improve the stability of the video and achieve smoothness. Video to improve user viewing experience.

In the first aspect, an embodiment of the present invention provides an image processing method, including:

Acquiring a first video, where the first video includes a plurality of first image frames;

Selecting at least one target image frame from the plurality of first image frames;

Acquiring posture data of the first image frame during exposure;

The at least one target image frame is adjusted according to the posture data to obtain a second video.

In the second aspect, an embodiment of the present invention provides an image processing device, including: a memory and a processor,

The memory is used to store programs;

The processor is configured to execute a program stored in the memory, and when the program is executed, the processor is configured to:

Acquiring posture data of the first image frame during exposure;

In the third aspect, an embodiment of the present invention provides a movable platform, including:

body;

The power system configured on the fuselage is used to provide mobile power for the movable platform;

Image processing equipment for:

Acquiring posture data of the first image frame during exposure;

In a fourth aspect, an embodiment of the present invention provides an imaging device, including:

The first acquiring unit is configured to acquire a first video, where the first video includes a plurality of first image frames;

A selecting unit, configured to select at least one target image frame from the plurality of first image frames;

The second acquiring unit is configured to acquire the posture data of the first image frame during exposure;

The adjusting unit is configured to adjust the at least one target image frame according to the posture data to obtain a second video.

In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium in which a computer program is stored, and when the computer program is executed by a processor, the method described in the first aspect is implemented. step.

The embodiment of the present invention can obtain a first video, select at least one target image frame from a plurality of first image frames included in the first video, and obtain the posture data of the first image frame at the time of exposure, so as to adjust at least One target image frame obtains the second video. Through this implementation manner, the phenomenon of jitter in the second video can be avoided, which helps to improve the stability of the video, realizes a smooth video, and improves the user's viewing experience.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative labor, other drawings can be obtained from these drawings.

Fig. 1 is a schematic structural diagram of an image processing system provided by an embodiment of the present invention;

2 is a schematic flowchart of an image processing method provided by an embodiment of the present invention;

3 is a schematic flowchart of another image processing method provided by an embodiment of the present invention;

4 is a schematic structural diagram of an image processing device provided by an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of an imaging device provided by an embodiment of the present invention.

Detailed ways

The following clearly describes the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail with reference to the drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

At present, most shooting devices (such as (sports) cameras, cameras, handheld pan-tilts, etc.) are equipped with EIS algorithms, and the EIS algorithm can perform posture correction on the videos (or images) acquired by the shooting devices. For example, after a frame of image is exposed, the EIS algorithm can perform low-pass filtering on the posture data of the camera collected by the gyroscope for a period of time before the exposure to generate a smooth curve (ie, posture curve). Then, the EIS algorithm can calculate the posture compensation amount according to the posture curve, and correct the video (or image) based on the posture compensation amount, so that the user can watch a relatively smooth video (or image) picture.

In one embodiment, on the basis of the above EIS algorithm, the camera equipped with the EIS algorithm can make full use of the inertial measurement unit (IMU) of the image before and after the exposure by delaying the processing of the image. ) Data, and integrate the accelerometer and gyroscope data, and perform a frequency-domain low-pass filter on the posture data of the camera to obtain a smooth curve that can reflect the posture change of the camera.

In one embodiment, although the video obtained by the EIS algorithm has stabilization, if the video is accelerated by N times to implement mobile time-lapse photography, the video jitter will be very serious. Generally, a low-pass filter with a cut-off frequency of about 1 Hz is sufficient for video stabilization. If the video is accelerated to 40 times, it will become jitter with a cut-off frequency of about 40 Hz, which looks very jittery. If you continue to increase the memory depth of the First Input First Output (FIFO), the video acceleration to 40 times requires about 900 images of 40s to be cached, and several gigabytes of memory are required, which will cause the video screen to jitter or twitch, and The temporary memory usage rate is high, which reduces the viewing experience.

The embodiment of the present invention provides an image processing method that saves memory by extracting frames first and then buffering the image. At the same time, the ultra-long IMU data is cached, the image frames are adjusted according to the IMU data, and invalid image frames are discarded. In this case, realize super smooth moving time-lapse video to avoid video jitter.

The image processing method provided in the embodiment of the present invention may be executed by an image processing device in an image processing system, where the image processing system may include a photographing device and an image processing device. In some embodiments, the image processing device may be provided on the photographing device; in some embodiments, the image processing device may be spatially independent of the photographing device. In some embodiments, the photographing device and the image processing device establish a communication connection in a wired or wireless manner. In some embodiments, the camera may be mounted on a movable platform; in some embodiments, the movable platform includes, but is not limited to, unmanned aerial vehicles, unmanned vehicles, movable robots, etc. equipment. In some embodiments, the photographing device includes, but is not limited to, a camera, a camera, a handheld pan-tilt, an image acquisition sensor, and the like. In some embodiments, the image processing device may include one or more of a smart phone, a tablet computer, a laptop computer, and a wearable device.

The image processing system provided by the embodiment of the present invention will be schematically illustrated below by taking FIG. 1 as an example.

Please refer to FIG. 1. FIG. 1 is a schematic structural diagram of an image processing system according to an embodiment of the present invention. The image processing system includes a photographing device 11 and an image processing device 12. In some embodiments, the image processing device 12 is spatially independent of the photographing device 11, and in some embodiments, a communication connection can be established between the photographing device 11 and the image processing device 12. In some embodiments, the image processing device 12 may include one or more of a smart phone, a tablet computer, a laptop computer, and a wearable device. In some embodiments, the photographing device 11 may be mounted on a movable platform 13. In some embodiments, the movable platform 13 includes a power system 131, and the power system 131 is configured to be movable. The platform 13 provides power for movement.

In the embodiment of the present invention, the image processing device 12 may obtain the first video shot by the shooting device 11, select at least one target image frame from a plurality of first image frames included in the first video, and obtain the first video frame. The posture data of the image frame during exposure, so as to adjust the at least one target image frame according to the posture data to obtain the second video. Through this implementation manner, the phenomenon of video jitter can be avoided, which helps to further improve the stability of the video, realize a smooth video, and improve the user's viewing experience.

The image processing method provided by the embodiment of the present invention will be schematically described below in conjunction with the accompanying drawings.

Please refer to FIG. 2 for details. FIG. 2 is a schematic flowchart of an image processing method according to an embodiment of the present invention. The method may be executed by an image processing device, and the specific explanation of the image processing device is as described above. Specifically, the method of the embodiment of the present invention includes the following steps.

S201: Acquire a first video, where the first video includes multiple first image frames.

In the embodiment of the present invention, the image processing device may obtain a first video shot by a shooting device, and the first video includes a plurality of first image frames. The explanation of the shooting device is as described above, and will not be repeated here. . By acquiring the first video including multiple first image frames, it helps to ensure that there is sufficient delay, which facilitates the subsequent filtering by the frequency domain low-pass filter.

In some embodiments, the first video may be a video stabilized by the EIS algorithm in the shooting device. In some embodiments, the image processing device may also acquire the exposure time of the first video when acquiring the first video. The exposure time helps to subsequently acquire the first posture data of the camera before the exposure and the camera after the exposure. The second posture data.

In one embodiment, before acquiring the first video, the image processing device may first acquire a preset number of first image frames collected by the shooting device, so as to ensure sufficient delay and facilitate filtering by the frequency domain low-pass filter . In some embodiments, the preset number can be preset. For example, the first video composed of the first image frames within 1 second is acquired, and if the capture frequency of the shooting device is 40 fps, the preset number is 40 frames; If the capture frequency of the camera is 60 fps, the preset number is 60 frames.

In other embodiments, the preset number can also be associated with a frequency-domain low-pass filter preset in the camera. For example, if the frequency-domain low-pass filter takes a short time to filter the posture data of the camera, it can be Properly increase the preset number; if the frequency domain low-pass filter takes a long time to filter the posture data of the camera, the preset number can be appropriately decreased. Therefore, the preset number in the embodiment of the present invention can be adjusted according to specific scenarios, and is not specifically limited here.

In some embodiments, the frequency domain low-pass filter includes but is not limited to at least one of the following: FIR filter and IIR filter. It should be noted that technical personnel can select a suitable frequency domain low-pass filter according to a specific scenario, and if the frequency domain low-pass filter can be realized, the corresponding filter falls within the protection scope of the present application.

S202: Select at least one target image frame from the multiple first image frames.

In the embodiment of the present invention, the image processing device may select at least one target image frame from the plurality of first image frames.

In one embodiment, when the image processing device selects at least one target image frame from the plurality of first image frames, it may select the at least one target image frame from the plurality of first image frames according to the playback frequency Exemplarily, the playing frequency may be a playing speed ratio of the second video to the first video. In some embodiments, the playing frequency is a variable frequency.

For example, assuming that the playback frequency is N times, then one target image frame can be selected at intervals of N first image frames in the first video. Exemplarily, when the playback frequency is 2 times, one target image frame can be selected at intervals of two first image frames in the first video, that is, when the first image frame is 40 frames, the target image frame is 20 frame.

In an embodiment, one or more first-in-first-out memories may be provided in the image processing device, and the image processing device may store the selected target image frame in a first-in-first-out memory of a specified depth for subsequent processing. In an example, the image processing device may store the selected target image frame in the first first-in first-out memory with a designated depth of 20 frames. In some embodiments, the specified depth of the first first-in first-out memory may match the number of target image frames, or may be greater than the number of target image frames.

It can be seen that by selecting at least one target image frame from a plurality of first image frames according to the playback frequency, it is helpful to match the depth of the first-in-first-out memory and save memory; in addition, it can also be used when shooting the first video. The jitter is filtered to make the output video smoother and effectively improve the viewing experience.

S203: Obtain posture data of the first image frame during exposure.

In the embodiment of the present invention, the image processing device may obtain the posture data of the first image frame during exposure. By acquiring the posture data of the first image frame at the time of exposure, it can be ensured that a better smooth curve is established to stabilize the target image frame, and the viewing effect of the output video is ensured.

In one embodiment, the image processing device may obtain the posture data of the multiple first image frames collected by the camera through the posture sensor during exposure. In some embodiments, the posture data is posture data of the camera when the camera takes an image. In some embodiments, the attitude sensor may include, but is not limited to, a three-axis gyroscope, a three-axis accelerometer, a three-axis electronic floppy disk, GPS, etc., which are not limited herein. In some embodiments, the attitude data includes angular acceleration and/or angular velocity.

In an embodiment, the image processing device may acquire the posture data of the plurality of first image frames periodically collected by the camera through the posture sensor during exposure. In some embodiments, the period may be associated with the period of the photographing device collecting multiple first image frames, for example, 400 posture data are collected within 1 second, and 40 first image frames are collected. In other embodiments, the period during which the posture sensor collects the posture data may also have nothing to do with the period during which the photographing device collects the first image frame, which is not limited here.

In some embodiments, the target image frame includes a first preset number of image frames, the posture data includes a second preset number of posture data, and the number of image frames corresponding to the second preset number is the The first preset number corresponds to more than twice the number of image frames. In an example, assuming that the image processing device extracts 20 target image frames, the posture data of the imaging device at the time of exposure of the first image frame including the target image frame can be acquired for more than 40 frames.

In some embodiments, when the target image frame exceeds a first preset number, the previous data in the target image frame is discarded. For example, assuming that the first preset number is 20 frames, when the target image frame is acquired When more than 20 frames are 21 frames, the data of the first target image frame is discarded to keep the target image frame as the first preset number, that is, 20 frames. In some embodiments, when the posture data exceeds the second preset number, the previous data in the posture data is discarded. For example, assuming that the second preset number is 200, when the acquired posture data exceeds 200 When the number is 201, the first posture data is discarded to keep the posture data at the second preset number, which is 200; for another example, assume that the second preset number of posture data corresponds to 20 image frames, which are captured by the camera When the posture data of the 21st frame image frame, the posture data of the first target image frame collected by the photographing device is discarded, so as to keep the posture data at the second preset quantity. Through this implementation manner, it is possible to avoid deleting the newly acquired posture data, and to ensure the validity of the posture data of the photographing device.

In one embodiment, when acquiring the posture data of the first image frame during exposure, the image processing device may acquire the first posture data of the photographing device and the first posture data of the photographing device before the first image frame is exposed. The second posture data of the photographing device after the image frame is exposed. In an embodiment, the image processing device may acquire the first posture data of the photographing device before the first image frame is exposed and the first image frame after the exposure time of the first image frame. The second posture data of the photographing device. Through this embodiment, the time span of the posture data can be increased, and it is ensured that the low-frequency movement of the camera after exposure does not affect the posture data of the camera.

In an embodiment, the posture data further includes third posture data corresponding to the exposure time of the first image frame. In this case, the processor may merge the third posture data into the first posture data and the second posture data. In the first posture data, the third posture data is used as the last posture data. In the second posture data, the third posture data is the last posture data. The three posture data are used as the first posture data. Of course, when the number of the first posture data and the second posture data is large, the third posture data may not be used. Technicians can make adjustments according to specific scenarios, which is not limited here.

In some embodiments, the first posture data may be one or more, the second posture data may be one or more, and the first posture data and the second posture data are stored in a second first-in first Out of memory. In some embodiments, the capacity of the second first-in first-out memory is more than twice that of the first first-in first-out memory.

In some embodiments, each target image frame corresponds to a set of posture data (that is, the first posture data and the second posture data). Therefore, each target image frame can be cached through the first first-in first-out memory while the posture The storage address of the data in the second first-in first-out memory is used as the characteristic data corresponding to each target image frame. Through this embodiment, the image processing device can read the posture data from the corresponding storage address in the second first-in first-out memory when reading the target image frame of each frame from the first first-in first-out memory, thereby improving the reading efficiency.

In one embodiment, for two adjacent target image frames, the second posture data of the previous target image frame and the first posture data of the next target image frame may not overlap, thereby reducing the amount of data calculation. Alternatively, the second posture data of the target image frame of the previous frame and the first posture data of the target image frame of the next frame overlap, so as to ensure that the second video obtained subsequently is smoother and more stable.

In one embodiment, when the image processing device acquires the first posture data of the photographing device before the first image frame is exposed and the second posture data of the photographing device after the first image frame is exposed, It is possible to obtain the IMU data when the photographing device photographs the first image frame, and to determine the first posture data of the photographing device and the first image before the first image frame is exposed according to the IMU data After the frame is exposed, the second posture data of the camera is completed.

S204: Adjust the at least one target image frame according to the posture data to obtain a second video.

In the embodiment of the present invention, the image processing device may adjust the at least one target image frame according to the posture data to obtain the second video.

In one embodiment, when adjusting the at least one target image frame according to the posture data, the image processing device may filter the posture data of the at least one target image frame according to a cutoff frequency to obtain target posture data. In some embodiments, the cut-off frequency is less than 1 Hz, in one example, the cut-off frequency is 0.5 Hz; in one example, the cut-off frequency is 0.025 Hz. By adopting a low cut-off frequency, it is helpful to subsequently filter the posture data of the target image frame through a frequency-domain low-pass filter, and improve the filtering effect of image jitter. It is understandable that the lower the cut-off frequency of the frequency domain low-pass filter, the stronger its ability to filter out high-frequency signals in the attitude data, and the smoother the target attitude obtained, that is, the slower the motion state of the image acquisition device. The lower the impact on the subsequent display of the video image.

In an embodiment, the image processing device may use the first posture data and the second posture data as input data of a preset frequency domain low-pass filter, and the frequency domain low-pass filter filters the first posture data and the second posture data. Two high-frequency signals in the attitude data that are greater than the cut-off frequency to obtain low-frequency signals that do not exceed the cut-off frequency in the first attitude data and the second attitude data. It is understandable that if the frequency-domain low-pass filter has the function of frequency-domain transformation, the first attitude data and the second attitude data can be directly input to the frequency-domain low-pass filter; if the frequency-domain low-pass filter does not have The function of frequency domain transformation requires frequency domain transformation of the first attitude data and the second attitude data, and then the first attitude data and the second attitude data after the frequency domain transformation are input to the frequency domain low-pass filter. Among them, the frequency domain transform method can refer to related technologies, which is not limited here.

In one embodiment, the image processing device may generate target posture data of the photographing device during exposure based on the low-frequency signal that does not exceed the cut-off frequency in the first posture data and the second posture data.

In one embodiment, when adjusting the at least one target image frame according to the posture data, the image processing device may perform integration processing on the posture data of the at least one target image frame to obtain estimated posture data. Wherein, the integration processing method is an existing integration processing technology, which will not be repeated here. Obtaining the estimated posture data through calculation helps to determine whether the adjustment of the target image frame is effective according to the estimated posture data and the target posture data.

In one embodiment, when the image processing device adjusts the at least one target image frame according to the posture data, it may determine whether the at least one target image frame is adjusted according to the estimated posture data and the target posture data. effective.

In one embodiment, when the image processing device determines whether the adjustment of the at least one target image frame is effective according to the estimated posture data and the target posture data, it may be based on the estimated posture data and the target posture data. Determine whether the adjustment of the at least one target image frame is effective, and if the difference exceeds the range, discard the estimated posture data whose difference exceeds the range and the target image frame corresponding to the target posture data .

In one embodiment, when the image processing device determines whether the adjustment of the at least one target image frame is effective according to the difference between the estimated posture data and the target posture data, it may acquire the estimated posture data and the target posture data. The posture difference of the target posture data. If the posture difference is less than or equal to the preset angle threshold, it can be determined that the adjustment of the at least one target image frame is effective. If the posture difference is greater than the preset angle threshold, then It is determined that the adjustment of the at least one target image frame is invalid. By determining the invalid target image frame through this implementation manner, the invalid target image frame can be discarded, which helps to improve the effectiveness of the target image frame, so that the second video obtained after adjustment is smoother and more stable.

For example, assuming that the preset angle threshold is k, if the estimated posture data of a target image frame is obtained as m1, the target posture data is n1, and the posture difference between m1 and n1 is m1-n1<k, then the target can be determined The adjustment of the image frame is effective.

For another example, assuming that the preset angle threshold is k, if the estimated posture data of a target image frame is obtained as m2, the target posture data is n2, and the posture difference between m2 and n2 is m2-n2>k, then the one can be determined The adjustment of the target image frame is invalid, and the one target image frame is discarded.

The embodiment of the present invention can obtain a first video, select at least one target image frame from a plurality of first image frames included in the first video, and obtain the posture data of the first image frame at the time of exposure, so as to adjust according to the posture data At least one target image frame obtains the second video. Through this implementation manner, unstable phenomena such as jitter in the second video can be avoided, which helps to improve the stability of the video, realize a smooth video, and improve the user's viewing experience.

Please refer to FIG. 3 for details. FIG. 3 is a schematic flowchart of another image processing method according to an embodiment of the present invention. As shown in FIG. 3, the image processing device can input the first video, select the target image frame according to the first video, and obtain the IMU data when the shooting device shoots the target image frame, and obtain the posture data of the shooting device according to the IMU data, thereby Determine the estimated posture data and the target posture data according to the posture data. When it is determined that the posture difference between the estimated posture data and the target posture data is less than or equal to the preset angle threshold, it is determined that the adjustment of the target image frame is effective, and further the target image frame Filter to output the second video. Through this implementation manner, stabilization of the input first video is realized, and the stability of the output second video is improved, so that a more stable video can be displayed.

The embodiment of the present invention can obtain a first video, select at least one target image frame from a plurality of first image frames included in the first video, and obtain the posture data of the first image frame at the time of exposure, so as to adjust at least One target image frame obtains the second video. Through this implementation manner, unstable phenomena such as jitter in the second video can be avoided, which helps to improve the stability of the video, realize a smooth video, and improve the user's viewing experience.

Please refer to FIG. 4, which is a schematic structural diagram of an image processing device according to an embodiment of the present invention. Specifically, the image processing device includes: a memory 401 and a processor 402.

In an embodiment, the image processing device further includes a data interface 403, and the data interface 403 is used to transfer data information between the image processing device and other devices.

The memory 401 may include a volatile memory (volatile memory); the memory 401 may also include a non-volatile memory (non-volatile memory); the memory 401 may also include a combination of the foregoing types of memories. The processor 402 may be a central processing unit (CPU). The processor 402 may further include a hardware chip. The above-mentioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), or any combination thereof.

The memory 401 is used to store programs, and the processor 402 can call the programs stored in the memory 401 to perform the following steps:

Acquiring posture data of the first image frame during exposure;

Further, when the processor 402 selects at least one target image frame from the plurality of first image frames, it is specifically configured to:

The at least one target image frame is selected from the plurality of first image frames according to a playing frequency, and the playing frequency is a ratio of the playing speed of the second video to the first video.

Further, the playing frequency is a variable frequency.

Further, when the processor 402 obtains the posture data of the first image frame during exposure, it is specifically configured to:

Acquire first posture data of the photographing device before the first image frame is exposed and second posture data of the photographing device after the first image frame is exposed.

Further, when the processor 402 obtains the first posture data of the photographing device before the exposure of the first image frame and the second posture data of the photographing device after the first image frame is exposed, it specifically uses in:

Acquiring IMU data when the photographing device photographed the first image frame;

Determine the first posture data of the photographing device before the first image frame is exposed and the second posture data of the photographing device after the first image frame is exposed according to the IMU data.

Further, when the processor 402 adjusts the at least one target image frame according to the posture data, it is specifically configured to:

Filter the posture data of the at least one target image frame according to the cutoff frequency to obtain target posture data.

Further, the cut-off frequency is less than 1 Hz.

Performing integration processing on the posture data of the at least one target image frame to obtain estimated posture data.

According to the estimated posture data and the target posture data, it is determined whether the adjustment of the at least one target image frame is effective.

Further, when the processor 402 determines whether the adjustment of the at least one target image frame is effective according to the estimated pose data and the target pose data, it is specifically configured to:

Determining whether the adjustment of the at least one target image frame is effective according to the difference between the estimated posture data and the target posture data;

If the difference value is out of the range, discard the estimated posture data and the target image frame corresponding to the target posture data whose difference value is out of the range.

Further, when the processor 402 determines whether the adjustment of the at least one target image frame is effective according to the difference between the estimated pose data and the target pose data, it is specifically configured to:

Acquiring the posture difference between the estimated posture data and the target posture data;

If the attitude difference is less than or equal to the preset angle threshold, determining that the adjustment of the at least one target image frame is valid;

If the attitude difference is greater than the preset angle threshold, it is determined that the adjustment of the at least one target image frame is invalid.

Further, the posture data includes angular acceleration and/or angular velocity.

Further, the target image frame includes a first preset number of image frames, the posture data includes a second preset number of posture data, and the second preset number is twice the first preset number the above.

Further, when the target image frame exceeds a first preset number, discard the previous data in the target image frame;

When the posture data exceeds the second preset number, the previous data in the posture data is discarded.

Further, the first video is stored in a first first-in first-out memory of a specified depth.

Further, the first posture data is one or more, the second posture data is one or more, and the first posture data and the second posture data are stored in a second first-in first-out memory.

Further, the capacity of the second first-in first-out memory is more than twice that of the first first-in first-out memory.

Please refer to FIG. 5, which is a schematic structural diagram of an imaging device according to an embodiment of the present invention. Specifically, the image processing device includes: a first acquisition unit 501, a selection unit 502, a second acquisition unit 503, and an adjustment unit 504;

The first acquiring unit 501 is configured to acquire a first video, where the first video includes a plurality of first image frames;

The selecting unit 502 is configured to select at least one target image frame from the plurality of first image frames;

The second acquiring unit 503 is configured to acquire the posture data of the first image frame during exposure;

The adjusting unit 504 is configured to adjust the at least one target image frame according to the posture data to obtain a second video.

Further, when the selecting unit 502 selects at least one target image frame from the plurality of first image frames, it is specifically configured to:

The at least one target image frame is selected from the plurality of first image frames according to a playing frequency, where the playing frequency is a ratio of the playing speed of the second video to the first video.

Further, the playing frequency is a variable frequency.

Further, when the second acquiring unit 503 acquires the posture data of the first image frame during exposure, it is specifically used for:

Further, when the second acquiring unit 503 acquires the first posture data of the photographing device before the first image frame is exposed and the second posture data of the photographing device after the first image frame is exposed, Specifically used for:

Further, when the adjusting unit 504 adjusts the at least one target image frame according to the posture data, it is specifically configured to:

Further, the cut-off frequency is less than 1 Hz.

Further, when the adjustment unit 504 determines whether the adjustment of the at least one target image frame is effective according to the estimated pose data and the target pose data, it is specifically configured to:

If the difference value is out of the range, discarding the estimated posture data and the target image frame corresponding to the target posture data whose difference value is out of the range.

Further, when the adjustment unit 504 determines whether the adjustment of the at least one target image frame is effective according to the difference between the estimated pose data and the target pose data, it is specifically configured to:

The embodiment of the present invention also provides a movable platform, including: a fuselage; a power system configured on the fuselage to provide mobile power for the movable platform; an image processing device for acquiring the first video, so The first video includes a plurality of first image frames; at least one target image frame is selected from the plurality of first image frames; the posture data of the first image frame at the time of exposure is acquired; the posture data is adjusted according to the posture data At least one target image frame to obtain a second video.

Further, when the image processing device selects at least one target image frame from the plurality of first image frames, it is specifically configured to:

Further, the playing frequency is a variable frequency.

Further, when the image processing device acquires the posture data of the first image frame at the time of exposure, it is specifically used for:

Further, when the image processing device acquires the first posture data of the photographing device before the exposure of the first image frame and the second posture data of the photographing device after the first image frame is exposed, specifically using in:

Further, when the image processing device adjusts the at least one target image frame according to the posture data, it is specifically configured to:

Further, the cut-off frequency is less than 1 Hz.

Further, when the image processing device determines whether the adjustment of the at least one target image frame is effective according to the estimated pose data and the target pose data, it is specifically used to:

Further, when the image processing device determines whether the adjustment of the at least one target image frame is effective according to the difference between the estimated pose data and the target pose data, it is specifically used to:

The embodiment of the present invention also provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the embodiment corresponding to FIG. 2 of the present invention is implemented. , The device corresponding to the embodiment of the present invention described in FIG. 4 can also be implemented, which will not be repeated here.

The computer-readable storage medium may be an internal storage unit of the device described in any of the foregoing embodiments, such as a hard disk or memory of the device. The computer-readable storage medium may also be an external storage device of the device, such as a plug-in hard disk equipped on the device, a Smart Media Card (SMC), or a Secure Digital (SD) card. , Flash Card, etc. Further, the computer-readable storage medium may also include both an internal storage unit of the device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

The above-disclosed are only some embodiments of the present invention, which of course cannot be used to limit the scope of rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope of the present invention.

Claims

An image processing method, characterized in that it comprises:

Acquiring a first video, where the first video includes a plurality of first image frames;

Selecting at least one target image frame from the plurality of first image frames;

Acquiring posture data of the first image frame during exposure;

The at least one target image frame is adjusted according to the posture data to obtain a second video.
The method according to claim 1, wherein the selecting at least one target image frame from the plurality of first image frames comprises:

The at least one target image frame is selected from the plurality of first image frames according to a playing frequency, where the playing frequency is a ratio of the playing speed of the second video to the first video.
The method according to claim 2, wherein the playing frequency is a variable frequency.
The method according to claim 1, wherein the acquiring the posture data of the first image frame during exposure comprises:

Acquire first posture data of the photographing device before the first image frame is exposed and second posture data of the photographing device after the first image frame is exposed.
The method according to claim 4, wherein the acquiring the first posture data of the photographing device before the exposure of the first image frame and the first posture data of the photographing device after the first image frame is exposed 2. Posture data, including:

Acquiring IMU data when the photographing device photographed the first image frame;

Determine the first posture data of the photographing device before the first image frame is exposed and the second posture data of the photographing device after the first image frame is exposed according to the IMU data.
The method according to claim 1, wherein the adjusting the at least one target image frame according to the posture data comprises:

Filter the posture data of the at least one target image frame according to the cutoff frequency to obtain target posture data.
The method according to claim 6, wherein the cut-off frequency is less than 1 Hz.
The method according to claim 6, wherein the adjusting the at least one target image frame according to the posture data comprises:

Performing integration processing on the posture data of the at least one target image frame to obtain estimated posture data.
The method according to claim 8, wherein the adjusting the at least one target image frame according to the posture data comprises:

According to the estimated posture data and the target posture data, it is determined whether the adjustment of the at least one target image frame is effective.
The method according to claim 9, wherein the determining whether the adjustment of the at least one target image frame is effective according to the estimated pose data and the target pose data comprises:

Determining whether the adjustment of the at least one target image frame is effective according to the difference between the estimated posture data and the target posture data;

If the difference value is out of the range, discarding the estimated posture data and the target image frame corresponding to the target posture data whose difference value is out of the range.
The method according to claim 10, wherein the determining whether the adjustment of the at least one target image frame is effective according to the difference between the estimated pose data and the target pose data comprises:

Acquiring the posture difference between the estimated posture data and the target posture data;

If the attitude difference is less than or equal to the preset angle threshold, determining that the adjustment of the at least one target image frame is valid;

If the attitude difference is greater than the preset angle threshold, it is determined that the adjustment of the at least one target image frame is invalid.
The method according to claim 8, wherein the attitude data includes angular acceleration and/or angular velocity.
The method according to claim 1, wherein the target image frame includes a first preset number of image frames, the posture data includes a second preset number of posture data, and the second preset number is More than twice the first preset number.
The method of claim 13, wherein:

When the target image frame exceeds the first preset number, discard the previous data in the target image frame;

When the posture data exceeds the second preset number, the previous data in the posture data is discarded.
An image processing device, characterized by comprising: a processor and a memory,

The memory is used to store programs;

The processor is configured to execute a program stored in the memory, and when the program is executed, the processor is configured to:

Acquiring a first video, where the first video includes a plurality of first image frames;

Selecting at least one target image frame from the plurality of first image frames;

Acquiring posture data of the first image frame during exposure;

The at least one target image frame is adjusted according to the posture data to obtain a second video.
The device according to claim 15, wherein when the processor selects at least one target image frame from the plurality of first image frames, it is specifically configured to:

The at least one target image frame is selected from the plurality of first image frames according to a playing frequency, where the playing frequency is a ratio of the playing speed of the second video to the first video.
The device according to claim 16, wherein the playing frequency is a variable frequency.
The device according to claim 15, wherein when the processor obtains the posture data of the first image frame during exposure, it is specifically configured to:

Acquire first posture data of the photographing device before the first image frame is exposed and second posture data of the photographing device after the first image frame is exposed.
The apparatus according to claim 18, wherein the processor acquires the first posture data of the photographing device before the first image frame is exposed and the photographing device after the first image frame is exposed When the second posture data of is specifically used for:

Acquiring IMU data when the photographing device photographed the first image frame;

Determine the first posture data of the photographing device before the first image frame is exposed and the second posture data of the photographing device after the first image frame is exposed according to the IMU data.
The device according to claim 15, wherein when the processor adjusts the at least one target image frame according to the posture data, it is specifically configured to:

Filter the posture data of the at least one target image frame according to the cutoff frequency to obtain target posture data.
The device according to claim 20, wherein the cut-off frequency is less than 1 Hz.
The device according to claim 20, wherein when the processor adjusts the at least one target image frame according to the posture data, it is specifically configured to:

Performing integration processing on the posture data of the at least one target image frame to obtain estimated posture data.
The device according to claim 22, wherein when the processor adjusts the at least one target image frame according to the posture data, it is specifically configured to:

According to the estimated posture data and the target posture data, it is determined whether the adjustment of the at least one target image frame is effective.
The device according to claim 23, wherein the processor determines whether the adjustment of the at least one target image frame is effective according to the estimated posture data and the target posture data, which is specifically configured to:

Determining whether the adjustment of the at least one target image frame is effective according to the difference between the estimated posture data and the target posture data;

If the difference value is out of the range, discarding the estimated posture data and the target image frame corresponding to the target posture data whose difference value is out of the range.
The device according to claim 24, wherein the processor is specifically configured to determine whether the adjustment of the at least one target image frame is effective according to the difference between the estimated posture data and the target posture data. :

Acquiring the posture difference between the estimated posture data and the target posture data;

If the attitude difference is less than or equal to the preset angle threshold, determining that the adjustment of the at least one target image frame is valid;

If the attitude difference is greater than the preset angle threshold, it is determined that the adjustment of the at least one target image frame is invalid.
The device according to claim 22, wherein the posture data includes angular acceleration and/or angular velocity.
The device according to claim 15, wherein the target image frame includes a first preset number of image frames, the posture data includes a second preset number of posture data, and the second preset number is More than twice the first preset number.
The device of claim 27, wherein:

When the target image frame exceeds the first preset number, discard the previous data in the target image frame;

When the posture data exceeds the second preset number, the previous data in the posture data is discarded.
The device according to claim 15 or 16, wherein the first video is stored in a first first-in first-out memory of a specified depth.
The device according to claim 18 or 19, wherein the first posture data is one or more, the second posture data is one or more, and the first posture data and the second posture data are The posture data is stored in the second first-in first-out memory.
The device according to claim 30, wherein the capacity of the second first-in first-out memory is more than twice that of the first first-in first-out memory.
A movable platform, characterized in that it comprises:

body;

The power system configured on the fuselage is used to provide mobile power for the movable platform;

Image processing equipment for:

Acquiring a first video, where the first video includes a plurality of first image frames;

Selecting at least one target image frame from the plurality of first image frames;

Acquiring posture data of the first image frame during exposure;

The at least one target image frame is adjusted according to the posture data to obtain a second video.
The movable platform according to claim 32, wherein when the image processing device selects at least one target image frame from the plurality of first image frames, it is specifically configured to:

The at least one target image frame is selected from the plurality of first image frames according to a playing frequency, where the playing frequency is a ratio of the playing speed of the second video to the first video.
The movable platform according to claim 33, wherein the playing frequency is a variable frequency.
The movable platform according to claim 32, wherein when the image processing device acquires the posture data of the first image frame at the time of exposure, it is specifically used for:

Acquire first posture data of the photographing device before the first image frame is exposed and second posture data of the photographing device after the first image frame is exposed.
The movable platform according to claim 35, wherein the image processing device acquires the first posture data of the photographing device before the first image frame is exposed and the first image frame after the exposure. When describing the second posture data of the camera, it is specifically used for:

Acquiring IMU data when the photographing device photographed the first image frame;

Determine the first posture data of the photographing device before the first image frame is exposed and the second posture data of the photographing device after the first image frame is exposed according to the IMU data.
The movable platform according to claim 32, wherein when the image processing device adjusts the at least one target image frame according to the posture data, it is specifically configured to:

Filter the posture data of the at least one target image frame according to the cutoff frequency to obtain target posture data.
The movable platform of claim 37, wherein the cut-off frequency is less than 1 Hz.
The movable platform according to claim 37, wherein when the image processing device adjusts the at least one target image frame according to the posture data, it is specifically configured to:

Performing integration processing on the posture data of the at least one target image frame to obtain estimated posture data.
The movable platform according to claim 39, wherein when the image processing device adjusts the at least one target image frame according to the posture data, it is specifically configured to:

According to the estimated posture data and the target posture data, it is determined whether the adjustment of the at least one target image frame is effective.
The mobile platform according to claim 40, wherein the image processing device determines whether the adjustment of the at least one target image frame is effective according to the estimated posture data and the target posture data, which is specifically used for :

Determining whether the adjustment of the at least one target image frame is effective according to the difference between the estimated posture data and the target posture data;

If the difference value is out of the range, discard the estimated posture data and the target image frame corresponding to the target posture data whose difference value is out of the range.
The mobile platform according to claim 41, wherein the image processing device determines whether the adjustment of the at least one target image frame is effective according to the difference between the estimated posture data and the target posture data, Specifically used for:

Acquiring the posture difference between the estimated posture data and the target posture data;

If the attitude difference is less than or equal to the preset angle threshold, determining that the adjustment of the at least one target image frame is valid;

If the attitude difference is greater than the preset angle threshold, it is determined that the adjustment of the at least one target image frame is invalid.
The movable platform according to claim 39, wherein the posture data includes angular acceleration and/or angular velocity.
The movable platform according to claim 32, wherein the target image frame includes a first preset number of image frames, the posture data includes a second preset number of posture data, and the second preset The number is more than twice the first preset number.
The movable platform of claim 40, wherein:

When the target image frame exceeds the first preset number, discard the previous data in the target image frame;

When the posture data exceeds the second preset number, the previous data in the posture data is discarded.
The movable platform according to claim 32 or 33, wherein the first video is stored in a first-in-first-out memory of a specified depth.
The mobile platform according to claim 35 or 36, wherein the first posture data is one or more, the second posture data is one or more, the first posture data and the The second posture data is stored in the second first-in first-out memory.
An imaging device, characterized in that it comprises:

The first acquiring unit is configured to acquire a first video, where the first video includes a plurality of first image frames;

A selecting unit, configured to select at least one target image frame from the plurality of first image frames;

The second acquiring unit is configured to acquire the posture data of the first image frame during exposure;

The adjusting unit is configured to adjust the at least one target image frame according to the posture data to obtain a second video.
The device according to claim 48, wherein when the selecting unit selects at least one target image frame from the plurality of first image frames, it is specifically configured to:

The at least one target image frame is selected from the plurality of first image frames according to a playing frequency, where the playing frequency is a ratio of the playing speed of the second video to the first video.
The device of claim 49, wherein the playing frequency is a variable frequency.
The device according to claim 48, wherein when the second acquiring unit acquires the posture data of the first image frame during exposure, it is specifically configured to:

Acquire first posture data of the photographing device before the first image frame is exposed and second posture data of the photographing device after the first image frame is exposed.
The device according to claim 51, wherein the second acquiring unit acquires the first posture data of the photographing device before the first image frame is exposed, and the first image frame is When the second posture data of the photographing device is specifically used for:

Acquiring IMU data when the photographing device photographed the first image frame;

Determine the first posture data of the photographing device before the first image frame is exposed and the second posture data of the photographing device after the first image frame is exposed according to the IMU data.
The device according to claim 48, wherein when the adjustment unit adjusts the at least one target image frame according to the posture data, it is specifically configured to:

Filter the posture data of the at least one target image frame according to the cutoff frequency to obtain target posture data.
The device of claim 53, wherein the cut-off frequency is less than 1 Hz.
The device according to claim 53, wherein when the adjustment unit adjusts the at least one target image frame according to the posture data, it is specifically configured to:

Performing integration processing on the posture data of the at least one target image frame to obtain estimated posture data.
The device according to claim 55, wherein when the adjustment unit adjusts the at least one target image frame according to the posture data, it is specifically configured to:

According to the estimated posture data and the target posture data, it is determined whether the adjustment of the at least one target image frame is effective.
The device according to claim 56, wherein when the adjustment unit determines whether the adjustment of the at least one target image frame is effective according to the estimated pose data and the target pose data, it is specifically configured to:

Determining whether the adjustment of the at least one target image frame is effective according to the difference between the estimated posture data and the target posture data;

If the difference value is out of the range, discarding the estimated posture data and the target image frame corresponding to the target posture data whose difference value is out of the range.
The device according to claim 57, wherein the adjustment unit is specifically configured to determine whether the adjustment of the at least one target image frame is effective according to the difference between the estimated posture data and the target posture data. :

Acquiring the posture difference between the estimated posture data and the target posture data;

If the attitude difference is less than or equal to the preset angle threshold, determining that the adjustment of the at least one target image frame is valid;

If the attitude difference is greater than the preset angle threshold, it is determined that the adjustment of the at least one target image frame is invalid.
The device according to claim 55, wherein the posture data includes angular acceleration and/or angular velocity.
The device according to claim 48, wherein the target image frame comprises a first preset number of image frames, the posture data comprises a second preset number of posture data, and the second preset number is More than twice the first preset number.
The device of claim 60, wherein:

When the target image frame exceeds the first preset number, discard the previous data in the target image frame;

When the posture data exceeds the second preset number, the previous data in the posture data is discarded.
The device according to claim 48 or 49, wherein the first video is stored in a first first-in first-out memory of a specified depth.
The device according to claim 51 or 52, wherein the first posture data is one or more, the second posture data is one or more, and the first posture data and the second posture data are The posture data is stored in the second first-in first-out memory.
The device according to claim 63, wherein the capacity of the second first-in first-out memory is more than twice that of the first first-in first-out memory.
A computer-readable storage medium in which a computer program is stored, characterized in that: when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 14 are implemented .