WO2017075788A1 - Anti-jitter photographing method and apparatus, and camera device - Google Patents

Abstract

Embodiments of the present invention provide an anti-jitter photographing method and apparatus, and a camera device. The method comprises: obtaining an output signal of at least one inertial sensor corresponding to each of N frame images, wherein N≥2; calculating a jitter amplitude value of each of the N frame images according to the output signal of the at least one inertial sensor corresponding to each of the N frame images; selecting K frame target images according to the jitter amplitude value of each of the N frame images, wherein 2≤K≤N; calculating position offsets of the K frame target images relative to each other; aligning the K frame target images according to the position offsets of the K frame target images relative to each other, and combining the aligned K frame target images to generate a final image. The method, apparatus, and camera device of the embodiments of the present invention can effectively suppress image blurring caused by jitter, thereby improving the photographing experience of users.

Description

Anti-shake photographing method, device and camera device Technical field

The present invention relates to the field of communications technologies, and in particular, to an anti-shake photographing method, apparatus, and camera device.

Background technique

With the popularity of card machines, SLR cameras, micro-cameras, and smartphones and tablets with camera functions, more and more people are enjoying the fun of taking pictures. For most non-professional photographers, when shooting, often the image is blurred due to the shaking of the hand when the shutter is pressed, which causes problems for users who have certain requirements on the clarity of the photo.

Summary of the invention

The embodiment of the invention provides an anti-shake photographing method, a device and a camera device, which can effectively suppress image blur caused by shaking and improve the user's photographing experience.

An embodiment of the present invention provides an anti-shake photographing method, including: acquiring an output signal of at least one inertial sensor corresponding to each frame image in an N-frame image, N≥2; according to each of the N-frame images And calculating, by the output signal of the at least one inertial sensor corresponding to the frame image, a jitter amplitude value of each frame image in the N frame image; and selecting a K frame target image according to the jitter amplitude value of each frame image in the N frame image, 2≤K≤N; calculating a mutual positional shift of the K-frame target image; aligning the K-frame target image according to mutual positional shift of the K-frame target image and synthesizing the aligned K-frame target image to Generate the final image.

Another embodiment of the present invention provides an apparatus for anti-shake photographing, comprising: an acquiring unit, configured to acquire an output signal of at least one inertial sensor corresponding to each frame image in an N-frame image, N≥2; a calculating unit, configured to calculate, according to an output signal of the at least one inertial sensor corresponding to each frame image in the N frame image, a jitter amplitude value of each frame image in the N frame image; a target image selecting unit, configured to: Selecting a K frame target image according to a jitter amplitude value of each frame image in the N frame image, 2≤K≤N; a mutual position offset calculation unit, configured to calculate a phase of the K frame target image The mutual image offset unit is configured to align the K frame target image according to the mutual positional offset of the K frame target image and synthesize the aligned K frame target image to generate a final image.

Another embodiment of the present invention provides a camera device, including: at least one inertial sensor for detecting jitter of the camera device corresponding to each frame image in an N frame image, N≥2; a processor for Calculating a jitter amplitude value of each frame image in the N frame image according to an output signal of the at least one inertial sensor; selecting a K frame target image according to a jitter amplitude value of each frame image in the N frame image, 2 ≤ K ≤ N; calculating a mutual positional shift of the K-frame target image; aligning the K-frame target image according to a mutual positional shift of the K-frame target image and synthesizing the aligned K-frame target image to generate The final image.

It can be seen from the above technical solutions provided by the embodiments of the present invention that the method, the device, and the camera device of the embodiments of the present invention can effectively suppress image blur caused by jitter and improve the user's photographing experience.

DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description are only some of the present invention. For the embodiments, those skilled in the art can obtain other drawings according to the drawings without any creative work.

1 is a flowchart of an anti-shake photographing method according to an embodiment of the present invention;

2 is a flowchart of a method for acquiring angle jitter according to an embodiment of the present invention;

FIG. 3 is a flowchart of a method for acquiring shift jitter according to an embodiment of the present invention; FIG.

4 is a flow chart of a method for controlling an exposure start time according to an embodiment of the present invention;

FIG. 5 is a structural block diagram of an apparatus for anti-shake photographing according to an embodiment of the present invention; FIG.

6 is a schematic structural diagram of a camera device according to an embodiment of the present invention;

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

FIG. 1 is a flowchart of an anti-shake photographing method according to an embodiment of the present invention. The method provided in this embodiment can be applied to a digital camera, a digital image recorder, a smart phone, a monitor, and other electronic functions with camera functions. The embodiments of the present invention are not limited in the product. Please refer to Figure 1, including the following steps:

Step 101: Acquire an output signal of at least one inertial sensor corresponding to each frame image in the N frame image.

At present, there are many kinds of camera devices having a photographing function, such as a digital camera, a digital video camera, a mobile phone with a built-in camera, and a Personal Digital Assistant (PDA) having a built-in camera. The above camera device usually has an inertial sensor built in to measure acceleration or angular velocity. Inertial sensors include accelerometers and angular velocity sensors (such as gyroscopes) and their single, dual, and triaxial combination IMUs (Inertial Measurement Units).

Synthesizing the multi-frame image data can reduce the mutual positional shift of the multi-frame images displayed by each of the multi-frame image data obtained by time-division photography. In this embodiment, by acquiring the jitter amplitude value corresponding to each frame image in the N frame image, a K frame image with a small jitter amplitude is selected and a relatively clear image is synthesized, wherein N≥2, 2≤K≤ N.

Optionally, in the embodiment, the at least one inertial sensor comprises at least one of an angular velocity sensor and an acceleration sensor.

Step 102: Calculate a jitter amplitude value of each frame image in the N frame image.

Optionally, in this embodiment, the output signal of the at least one inertial sensor is Converted to a digital signal by an A/D converter.

Inertial sensor itself, carrier motion, external interference, installation process and many other factors will bring zero drift error to the output signal of the inertial sensor. Due to various reasons, this zero drift is often unavoidable.

In an alternative embodiment of the present embodiment, the output of the A/D converter is processed by a digital high pass filter to reduce errors due to zero drift.

In another alternative embodiment of the present embodiment, the output of the A/D converter is subjected to processing operations of the Kalman filter to reduce errors due to zero drift. The filtered signal is then subjected to an integral processing operation to calculate angular jitter or shift jitter.

If the inertial sensor is an angular velocity sensor, the output signal of the inertial sensor is an angular velocity signal, and the process of acquiring the angle jitter performed in the present embodiment is explained in conjunction with the flowchart shown in FIG. 2.

In step 201, the output signal of the angular velocity sensor is A/D converted, and the result of the A/D conversion is taken as ANG_VEL.

In step 202, a high-pass filtering operation (HPF) is performed on the calculation result of step 201 to reduce the error caused by the zero-point drift, and the filtered result is taken as ANG_VEL_HPF.

In step 203, an integration operation is performed on the calculation result of step 202, the integration result is DEG, and DEG is the first jitter angle displacement signal.

Optionally, in this embodiment, when the output signal of the at least one inertial sensor includes an angular velocity signal and an acceleration signal, if the angular velocity signal and the acceleration signal data acquisition time interval are not uniform, the method further includes before step 202. The interpolation result is performed on the calculation result of the step 201. The interpolation operation may be performed by a method such as a linear interpolation or a polynomial interpolation, which is not limited in the embodiment of the present invention.

If the inertial sensor is an accelerometer, the output signal of the inertial sensor is an acceleration signal, and the process of acquiring the shift jitter performed in the present embodiment is explained in conjunction with the flowchart shown in FIG.

In step 301, the output signal of the accelerometer is A/D converted, and the result of the A/D conversion is taken as ACC.

In step 302, a high-pass filtering operation is performed on the calculation result of step 301 to reduce the error caused by the zero point drift.

In step 303, an integration operation is performed on the calculation result of step 302, and the integration result is a signal indicating the speed V of the shift jitter.

In step 304, a high pass filtering operation is performed on the signal of the velocity V as the input shift jitter to reduce the error caused by the zero point drift.

In step 305, an integration operation is performed on the calculation result of step 304, and the integration result is DIS, and DIS is the first jitter shift signal.

Optionally, in this embodiment, when the output signal of the at least one inertial sensor includes an angular velocity signal and an acceleration signal, if the angular velocity signal and the acceleration signal data acquisition time interval are not uniform, the method further includes before step 302. The interpolation operation is performed on the calculation result of the step 301, and the interpolation operation may be performed by a method such as linear interpolation or polynomial interpolation, which is not limited in the embodiment of the present invention.

Optionally, in this embodiment, when the output signal of the at least one inertial sensor includes an angular velocity signal and an acceleration signal, the first jitter may be according to a relationship between the angle jitter, the shift jitter, and the object distance X. The shift signal DIS is converted into a first rotation signal ROT (where ROT=arcsin(DIS/X)), and then the first rotation signal ROT is vector-combined with the first dither angular displacement signal DEG to obtain a first target rotation signal DEG_TARGET And calculating a jitter amplitude value of each frame image in the N frame image according to the first target rotation signal DEG_TARGET.

After performing A/D conversion, filtering processing, and integration processing on the output signals of the at least one inertial sensor, each of the N frames of images may be calculated according to the filtering processing and the integrated processed signals (eg, DIS, DEG described above) The jitter amplitude value of the frame image.

In an optional implementation manner of this embodiment, the jitter amplitude value of each frame image in the N frame image may be generated according to a difference between the maximum value and the minimum value of the filtering process and the integrated processed signal.

In another optional implementation manner of this embodiment, the average value of the absolute value of the ratio of the difference between the signal value of the adjacent sample point and the time interval of the adjacent sample point after the filtering process and the integration process may be generated. The jitter amplitude value F _i of each frame image in the N frame image:

Where i represents the frame number, 1 ≤ _i ≤ N, η _i represents the number of sampling points of the ith frame, t _{i, j} represents the jth sampling time of the ith frame, and x _{i, j} represents the ith frame j sample point signal values.

In another optional implementation manner of this embodiment, the sequence H _i may be generated according to the filtering process and the integrated processed signal:

Any calculation of characteristic parameters of the sequence H _i, the characteristic values of the following parameters: the mean absolute value of the sequence H _i, H _i the sequence of the absolute value and then average the results of all the elements, the said standard deviation of the sequence H _i, H _i is the variance of the sequence;

Using the feature parameter as a jitter amplitude value of each frame image in the N frame image;

among them,

i represents the frame number, 1 ≤ _i ≤ N, η _i represents the number of sampling points of the ith frame, t _{i, j} represents the jth sampling time of the ith frame, and x _{i, j} represents the _{jth of} the ith frame Sample point signal value.

In another optional embodiment of the embodiment, the sequence A _{i is} generated according to the filtering process and the integrated processed signal:

among them,

x' _i,j =x _i,j -x _i,1

Where i represents the frame number, 1≤i≤N, 2≤k≤j, η _i represents the number of sampling points of the ith frame, and t _i,j represents the jth sampling time of the ith frame, x _i,j Indicates the jth sample point signal value of the ith frame, x' _i,j represents the difference between x _i,j and the initial sample point signal value x _i,1 , thereby eliminating the effect of the initial point on the result.

A(x _i,j ) represents an integral value of the ith ANG_VEL_HPF signal from t _i,1 to t _i,j in the i-th frame. In this embodiment, the ANG_VEL_HPF signal sample point signal value x _i,j and the initial sample point signal value x _{i , 1 is} used to eliminate the influence of the initial point on the result, and the integral value is calculated by the trapezoidal integral method.

Search for the peaks and troughs of the sequence A _i , assuming that the number of peaks in the sequence A(x _i,j ) is μ:

When μ=0, F(x _i )=max(A(x _i,j ))-min(A(x _i,j ));

Where max(A(x _i,j )) represents the maximum value of A(x _i,j ), and min(A(x _i,j )) represents the minimum value of A(x _i,j ).

When μ≥1, the position of the record peak is L _h , μ≥h≥1, and the jitter amplitude value of each frame image in the N frame image is generated according to the difference between the peak point and the adjacent valley point. ;

When μ=1,

When μ≥2, calculate the difference between the kth peak and the adjacent two troughs

Jitter amplitude value of the ith frame

Optionally, in this embodiment, when the output signal of the at least one inertial sensor includes an angular velocity signal and an acceleration signal, calculating a jitter amplitude of each frame image in the N frame image according to the first target rotation signal DEG_TARGET The method of the value is the same as the above-described method of calculating the jitter amplitude value of each frame image in the N frame image based on the filtering process and the integrated processed signal (for example, DIS, DEG).

Step 103: Select K according to the jitter amplitude value of each frame image in the N frame image. Frame target image.

In an optional implementation manner of this embodiment, the jitter amplitude values of each frame image in the N frame image are sorted, and the first K corresponding K frame images are taken as the target image in order from small to large. N≥K≥2.

In another optional implementation manner of this embodiment, the absolute value of the difference between the jitter amplitude value and the average value of each frame image in the N frame image is calculated, and then the absolute values of the difference values obtained above are sorted. The first K corresponding K frame images are taken as the target image in order from small to large, where N≥K≥2.

Step 104: Calculate a mutual position offset of the K frame target image.

In continuous shooting, due to the shaking of the human hand, even if the interval time of each frame is short, the subject may have a positional shift between the frames of the frame.

In an optional implementation manner of this embodiment, a motion vector indicating a mutual positional offset of the K-frame target image may be calculated according to an output signal of the at least one inertial sensor. The motion vector includes a plurality of components, each component being a time varying function or a time series for representing a component of translation and/or rotation of the image. For example, a first frame image of the N frame image may be selected as a reference frame image, and an output signal of the at least one inertial sensor corresponding to each frame image in the K frame target image and a reference frame image are corresponding. The output signal of the at least one inertial sensor calculates a positional offset of each frame image in the K frame target image with respect to the reference frame image, thereby obtaining a mutual positional offset of the K frame target image.

Optionally, any one of the filtered K frame target images may be selected as a reference frame image, and the output of the at least one inertial sensor corresponding to each frame image in the K frame target image is selected. And outputting, by the signal, the output signal of the at least one inertial sensor corresponding to the reference frame image, calculating a position offset of each frame image in the K frame target image with respect to the reference frame image, thereby obtaining a mutual position of the K frame target image Offset.

In another alternative embodiment of this embodiment, feature points may be extracted from the K-frame target image by using an image detection algorithm (eg, a Harris corner detection algorithm, a minimum equivalence segmentation absorption kernel (SUSAN) algorithm, etc.) Calculating a mutual positional offset of the feature points in the K-frame target image, the feature points in the K-frame target image The mutual positional offset is the mutual positional offset of the K-frame target image.

Step 105: Align the K frame target image according to mutual positional offset of the K frame target image and synthesize the aligned K frame target image to generate a final image.

In an optional implementation manner of this embodiment, the mutual position of the K frame target image is obtained according to the positional offset of each frame image in the K frame target image acquired in step 104 with respect to the reference frame image. Offset, resetting the position of each frame image in the K frame target image to align the K frame target image. In another optional implementation manner of the embodiment, the K-frame target image is reset according to the motion vector indicating the mutual positional deviation of the feature points in the K-frame target image acquired in step 104. The position of the feature point in each frame image is overlapped with each pixel point of the feature point in each frame image in the K frame target image to align the K frame target image.

In an optional implementation manner of this embodiment, after performing an alignment operation on the K frame target image, the K frame target image after performing the alignment operation may be synthesized according to a weighted average method, that is, performing an alignment operation The corresponding pixel values in the K-frame target image are summed and averaged to generate a final image, for example, the color value and the luminance value of each pixel of the K-frame target image after the alignment operation is performed may be summed The average is taken to generate the final image, and the color values include the color values of the red, green, and blue channels of the pixel.

Optionally, in the embodiment, before step 101, a control flow of the exposure start time of each frame of the N frame images is further included to minimize the jitter of the camera device during each frame image exposure. The exposure start time control flow executed in the present embodiment will now be explained in conjunction with the flowchart shown in FIG.

In step 401, an output signal of at least one inertial sensor within a preset period of time before the start of exposure corresponding to each frame image of the N frame image is detected.

In this embodiment, the preset time period before the start of the exposure may include the time of framing, focusing, and clicking the shutter. For example, it may be set to two seconds before the start of the exposure, which is not limited in this embodiment.

In step 402, a jitter prediction signal is generated.

In this embodiment, at least one inertial sensor output signal and each frequency component are detected for a period of time before the start of each frame image in the N frame image obtained in step 401. The weight and the exposure duration of each frame image in the N-frame image generate a jitter prediction signal for each of the N-frame images.

Optionally, in this embodiment, the inertial sensor includes at least one of an angular velocity sensor and an acceleration sensor, and in step 401, detecting a preset time period before the start of exposure corresponding to each frame image in the N frame image After the angular velocity signal output by the angular velocity sensor is performed, the processing flow of steps 201 to 203 is performed to obtain a second shake angle displacement signal DEG_DAT, and in step 401, the acceleration in the preset time period before the start of exposure corresponding to each frame image in the N frame image is detected. After the acceleration signal output by the sensor, the processing flow of steps 301 to 305 is performed to obtain the second jitter shift signal DIS_DAT.

Optionally, when the output signal of the at least one inertial sensor includes an angular velocity signal and an acceleration signal, the second jitter shift signal DIS_DAT may be converted into a relationship according to the angle jitter, the shift jitter, and the object distance X. The second rotation signal ROT_DAT (where ROT_DAT=arcsin(DIS_DAT/X)) performs vector synthesis of the second rotation signal ROT_DAT and the second dither angular displacement signal DEG_DAT to obtain a second target rotation signal DEG_TARGET_DAT.

Generally, the frequency of the jitter ranges from 0 Hz to 15 Hz. Therefore, in the present embodiment, the jitter angular displacement signal DEG_DAT and the jitter shift signal DIS_DAT are filtered by a band pass filter, optionally, in the present embodiment, band pass filtering The frequency range of the device is 0Hz to 15Hz.

In many cases, angular jitter (DEG_DAT), shift jitter (DIS_DAT), and second target rotation signal (DEG_TARGET_DAT) contain multiple frequency components, and the effects of different frequency components on imaging are different. Therefore, in the present embodiment, for a given exposure time length Δt, when the jitter prediction signal is to be generated, the degree of influence of each frequency component on the imaging is evaluated, and the weights assigned according to the evaluation result are combined to generate the jitter prediction. signal. Here, assigning weight means that when a plurality of components are combined to generate a combined value, each component is multiplied by a certain coefficient according to a predetermined criterion, that is, the frequency and the exposure time are fixed, before the calculation is performed, The table finds the weights corresponding to the respective frequency components, and generates a jitter prediction signal by using a weighted sum. The evaluation of the degree of influence of imaging is performed based on at least one of the angular velocity sensor output and the acceleration sensor output.

Since user jitter can be approximately decomposed into multiple sine or cosine waves of different amplitudes and frequencies, the following sine wave with period T, amplitude A, and initial phase 0

The method of calculating the weight is further introduced.

The weight is a function of the exposure duration Δt and the period T, expressed as θ(Δt, T). When the exposure start time is t and the exposure time is Δt, f(t) represents the angular range width of the rotation in the exposure period, t ∈ [0, T].

f(t) is related to the exposure time Δt. From the definition of the trigonometric function and the definition of f(t), f(t) is

Is a periodic function of the period, so only need to calculate f(t) at

The value above, the value of other intervals can be calculated periodically according to the function.

when

Time,

when

Time,

when

Time,

when

Time,

When Δt∈(T, +∞),

Optionally, in this embodiment, N discrete points may be uniformly selected on [0, T], the width of the angular range of each point in the exposure period is calculated, and the average is obtained to obtain weights:

In step 403, the exposure start time is controlled based on the jitter prediction signal and the exposure duration.

In an optional implementation manner of this embodiment, adjusting the N frame image according to a jitter prediction signal of each frame image in the N frame image and an exposure duration of each frame image in the N frame image. Exposure start time of each frame of image such that the jitter prediction signal of each frame image of the N frame images is at a maximum value and a minimum value within an exposure time period of each frame image of the N frame images The difference is the smallest.

In another optional implementation manner of this embodiment, the jitter prediction signal according to each frame image in the N frame image, the exposure duration of each frame image in the N frame image, and the hardware extension of the camera device Adjusting the exposure of each frame image in the N frame image a light start time such that a difference between a maximum value and a minimum value of a jitter prediction signal of each of the N frames of images in an exposure time of each of the N frames of images is minimized, wherein The hardware delay of the camera device is the difference between the time at which the sensor detects the value and the time at which the processor calculates the completion. Hardware delays vary depending on the device's specifications, including the time required for sensor detection, the time required to report to the processor, and the processor's computation time.

The method provided by the embodiment of the present invention can filter out the target image according to the output of the inertial sensor, align the target image based on the mutual position offset of the target image, and synthesize the target image to generate a final image, thereby effectively suppressing the jitter caused by The image is blurred to enhance the user's photo experience.

FIG. 5 is a structural block diagram of an apparatus for anti-shake photographing according to an embodiment of the present invention. Referring to FIG. 5, the apparatus includes:

The acquiring unit 51 is configured to acquire an output signal of at least one inertial sensor corresponding to each frame image in the N frame image, N≥2;

The jitter amplitude value calculation unit 52 is configured to calculate, according to an output signal of the at least one inertial sensor corresponding to each frame image in the N frame image acquired by the acquiring unit 51, a jitter amplitude of each frame image in the N frame image. value;

The target image selecting unit 53 is configured to select a K frame target image according to the jitter amplitude value of each frame image in the N frame image calculated by the jitter amplitude value calculating unit 52, 2≤K≤N;

a mutual position offset calculating unit 54 configured to calculate a mutual position offset of the K frame target image;

The final image generating unit 55 is configured to align the K-frame target image according to the mutual positional offset of the K-frame target image calculated by the mutual position offset calculating unit 54 and synthesize the aligned K-frame target image to generate The final image.

In an optional implementation manner of the embodiment, the jitter amplitude value calculation unit 52 may specifically include: a filter processing module 61, an integration processing module 62, and a calculation module 63, where the filter processing module 61 is used to acquire the acquisition unit 51. The output signal of the at least one inertial sensor corresponding to each frame image in the N frame image is subjected to filtering processing; the integration processing module 62 is configured to perform an integration process on the output signal of the filter processing module 61; The calculation module 63 is configured to calculate a jitter amplitude value of each frame image in the N frame image according to an output signal of the integration processing module 62.

In an optional implementation manner of the embodiment, the calculating module 63 may specifically generate the jitter of each frame image in the N frame image according to the difference between the maximum value and the minimum value of the output signal of the integration processing module 62. Amplitude value.

In another optional implementation manner of the embodiment, the calculating module 63 may specifically determine an absolute value of a ratio of a difference between adjacent sample point signal values and a time interval of adjacent sampling points according to an output signal of the integration processing module 62. The average value of the jitter amplitude value F _i of each of the N frames of images is generated:

Where i represents the frame number, 1 ≤ _i ≤ N, η _i represents the number of sampling points of the ith frame, t _{i, j} represents the jth sampling time of the ith frame, and x _{i, j} represents the ith frame j sample point signal values.

In another optional implementation manner of the embodiment, the calculating module 63 may specifically generate the sequence H _i according to the output signal of the integration processing module 62:

Any calculation of characteristic parameters of the sequence H _i, the characteristic values of the following parameters: the mean absolute value of the sequence H _i, H _i the sequence of the absolute value and then average the results of all the elements, the said standard deviation of the sequence H _i, H _i is the variance of the sequence;

Using the feature parameter as a jitter amplitude value of each frame image in the N frame image;

among them,

i represents the frame number, 1 ≤ _i ≤ N, η _i represents the number of sampling points of the ith frame, t _{i, j} represents the jth sampling time of the ith frame, and x _{i, j} represents the _{jth of} the ith frame Sample point signal value.

In another optional implementation manner of the embodiment, the calculating module 63 may specifically generate the sequence A _i according to the output signal of the integration processing module 62:

The peaks and troughs of the search sequence A _i are generated, and the jitter amplitude values of each of the N frames of images are generated based on the difference between the peak points and the adjacent valley points.

among them,

x _i,j ^′ =x _i,j -x _i,1

Where i represents the frame number, 1≤i≤N, 2≤k≤j, η _i represents the number of sampling points of the ith frame, and t _i,j represents the jth sampling time of the ith frame, x _i,j Indicates the value of the jth sample point signal of the ith frame.

In an optional implementation manner of the embodiment, the target image selecting unit 53 may specifically sort the jitter amplitude values of each frame image in the N frame image, and take the first K corresponding sequences in order from small to large. The K frame image is used as the K frame target image.

In this embodiment, the mutual position offset calculating unit 54 is specifically configured to calculate the K frame target image according to an output signal of at least one inertial sensor corresponding to each frame image in the N frame image acquired by the acquiring unit 51. Mutual positional offset; or, an image detection algorithm is used to extract feature points from the K-frame target image, and mutual positional offsets of the feature points in the K-frame target image are calculated.

In this embodiment, the at least one inertial sensor includes at least one of an angular velocity sensor and an acceleration sensor.

In an optional embodiment of the embodiment, the anti-shake photographing device further includes an exposure start time control unit 71 for controlling an exposure start time of each frame image in the N-frame image.

In an optional embodiment of the embodiment, the exposure start time control unit 71 includes a detection module 81, a generation module 82, and a control module 83. The detecting module 81 is configured to detect an output signal of at least one inertial sensor in a preset time period before the start of exposure corresponding to each frame image of the N frame image; and the generating module 82 is configured to detect according to the detecting module. Each of the N frames of images corresponds to an exposure before the start of the exposure The output signal of the at least one inertial sensor within the preset time period generates a jitter prediction signal of each frame image in the N frame image; the control module 83 is configured to use each frame in the N frame image generated by the generation module The exposure timing of the image of the jitter prediction signal of the image and each of the N frames of images controls the exposure start time of each of the N frames of images.

In an optional implementation manner of the embodiment, the generating module 82 is specifically configured to: output according to at least one inertial sensor in a preset period before the start of exposure corresponding to each frame image in the N frame image a signal, a weight of each frequency component, and an exposure duration of each of the N frames of images to generate a jitter prediction signal for each of the N frames of images, the weights of the respective frequency components being used to evaluate each The degree to which the output signal of the inertial sensor of the frequency component affects the imaging.

In an optional implementation manner of this embodiment, the control module 83 is specifically configured to: according to the jitter prediction signal of each frame image in the N frame image and each frame image in the N frame image. The exposure duration adjusts an exposure start time of each of the N frames of images such that an exposure prediction signal of each of the N frames of images is exposed in each of the N frames of images The difference between the maximum and minimum values in the duration is minimum.

The device provided by the embodiment of the present invention may filter out a target image according to an output of the inertial sensor, and align the target image based on mutual position offset of the target image and synthesize the target image to generate a final image, thereby effectively suppressing the jitter caused by The image is blurred to enhance the user's photo experience.

FIG. 6 is a schematic structural diagram of a camera device according to an embodiment of the present invention. In the embodiment of the present invention, the camera device may be a digital camera, a digital camera, a mobile phone with a built-in camera, and a personal digital assistant with a built-in camera (Personal) Digital Assistant, PDA for short. As shown in Figure 6, it generally includes at least one processor (e.g., a CPU) and at least one inertial sensor. It will be understood by those skilled in the art that the structure of the photographic apparatus shown in FIG. 6 does not constitute a limitation on an electronic apparatus, and the photographic apparatus may include more or less components than those illustrated, or may combine some components, or Different parts are arranged.

The specific components of the camera device will be specifically described below with reference to FIG. According to Figure 6, the camera device comprises:

At least one inertial sensor 601 is configured to detect jitter of the camera device corresponding to each frame image in the N frame image, N≥2;

The processor 602 is configured to calculate, according to an output signal of the at least one inertial sensor 601, a jitter amplitude value of each frame image in the N frame image; and a jitter amplitude value according to each frame image in the N frame image. Selecting a K-frame target image, 2≤K≤N; calculating a mutual positional offset of the K-frame target image; aligning the K-frame target image according to the mutual positional offset of the K-frame target image and synthesizing the aligned objects The K frame target image is described to generate a final image.

Communication bus 603 is used to implement connection communication between processor 602 and at least one inertial sensor 601.

In an optional implementation manner of this embodiment, the processor 602 is specifically configured to:

Performing filtering processing and integration processing on an output signal of at least one inertial sensor corresponding to each frame image in the N frame image;

A jitter amplitude value of each of the N frames of images is calculated based on the filtering process and the integrated processed signal.

In an optional implementation manner of this embodiment, the processor 602 is further configured to:

A jitter amplitude value of each of the N frames of images is generated based on a difference between a maximum value and a minimum value of the filter processed and integrated processed signals.

In another optional implementation manner of this embodiment, the processor 602 is further configured to:

Generating a jitter amplitude value of each frame image in the N frame image according to an average value of absolute values of a ratio of a difference between a signal value of a neighboring sample point signal and a time interval of adjacent sample points after the filtering process and the integration process F _i :

Where i represents the frame number, 1 ≤ _i ≤ N, η _i represents the number of sampling points of the ith frame, t _{i, j} represents the jth sampling time of the ith frame, and x _{i, j} represents the ith frame j sample point signal values.

In another optional implementation manner of this embodiment, the processor 602 is further configured to:

Generating sequence H _i according to the filtering process and the integrated processed signal:

Any calculation of characteristic parameters of the sequence H _i, the characteristic values of the following parameters: the mean absolute value of the sequence H _i, H _i the sequence of the absolute value and then average the results of all the elements, the said standard deviation of the sequence H _i, H _i is the variance of the sequence;

Using the feature parameter as a jitter amplitude value of each frame image in the N frame image;

among them,

i represents the frame number, 1 ≤ _i ≤ N, η _i represents the number of sampling points of the ith frame, t _{i, j} represents the jth sampling time of the ith frame, and x _{i, j} represents the _{jth of} the ith frame Sample point signal value.

In another optional implementation manner of this embodiment, the processor 602 is further configured to:

Generating sequence A _i according to the filtering process and the integrated processed signal:

The peaks and troughs of the search sequence A _i are generated, and the jitter amplitude values of each of the N frames of images are generated based on the difference between the peak points and the adjacent valley points.

among them,

x _i,j ^′ =x _i,j -x _i,1

Where i represents the frame number, 1≤i≤N, 2≤k≤j, η _i represents the number of sampling points of the ith frame, and t _i,j represents the jth sampling time of the ith frame, x _i,j Indicates the value of the jth sample point signal of the ith frame.

In this embodiment, the processor 602 is specifically configured to: sort the jitter amplitude values of each frame image in the N frame image, and take the first K corresponding K frame images in order from small to large. The K frame target image.

In this embodiment, the at least one inertial sensor comprises an angular velocity sensor and At least one of the speed sensors.

In this embodiment, the processor 602 is specifically configured to: calculate a mutual position offset of the K frame target image according to an output signal of the at least one inertial sensor 601 corresponding to each frame image in the N frame image; Alternatively, feature points are extracted from the K-frame target image using an image detection algorithm, and mutual positional offsets of the feature points in the K-frame target image are calculated.

In an optional implementation manner of this embodiment, the processor 602 is further configured to: acquire at least one inertial sensor in a preset time period before the start of exposure corresponding to each frame image in the N frame image Outputting a signal; generating, according to an output signal of the at least one inertial sensor in a preset period before the start of exposure corresponding to each frame image in the N frame image, a jitter prediction signal of each frame image in the N frame image; And an exposure start time of each of the N frames of images is controlled according to a jitter prediction signal of each of the N frames of images and an exposure duration of each of the N frames of images.

In an optional implementation manner of this embodiment, the processor 602 is specifically configured to: according to at least one inertial sensor in a preset time period before the start of exposure corresponding to each frame image in the N frame image The output signal, the weight of each frequency component, and the exposure duration of each of the N frames of images generate a jitter prediction signal for each of the N frames of images. The weight of each frequency component is used to evaluate the degree of influence of the output signal of the inertial sensor of each frequency component on imaging.

In an optional implementation manner of this embodiment, the processor 602 is further configured to: according to the jitter prediction signal of each frame image in the N frame image and each frame image in the N frame image The exposure duration adjusts an exposure start time of each of the N frames of images such that a jitter prediction signal of each of the N frames of images is in each of the N frames of images The difference between the maximum and minimum values within the exposure duration is minimal.

The camera device provided by the embodiment of the present invention can filter out the target image according to the output of the inertial sensor, align the target image based on the mutual positional offset of the target image, and synthesize the target image to generate a final image, thereby effectively suppressing the jitter caused. The image is blurred to enhance the user's photo experience.

Those skilled in the art can understand that all of the above embodiments are implemented or The partial process can be completed by a computer program to instruct related hardware. The above program can be stored in a computer readable storage medium, and when executed, the program can include the flow of an embodiment of each method as described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

The principles and embodiments of the present invention have been described herein with reference to specific examples. The foregoing description of the embodiments are merely for the purpose of understanding the method of the present invention and the concept thereof. Also, those skilled in the art in accordance with the present invention The present invention is not limited by the scope of the present invention.

Claims (37)

1. A method for anti-shake photographing, comprising:

   Obtaining an output signal of at least one inertial sensor corresponding to each frame image in the N frame image, N≥2;

   Calculating a jitter amplitude value of each frame image in the N frame image according to an output signal of at least one inertial sensor corresponding to each frame image in the N frame image;

   Selecting a K frame target image according to a jitter amplitude value of each frame image in the N frame image, 2≤K≤N;

   Calculating a mutual position offset of the K frame target image;

   The K frame target image is aligned according to mutual positional shift of the K frame target image and the aligned K frame target image is synthesized to generate a final image.
2. The method according to claim 1, wherein said calculating an image of each of said N frames of images based on an output signal of at least one inertial sensor corresponding to each of said N frames of images Amplitude values, including:

   Performing filtering processing and integration processing on an output signal of at least one inertial sensor corresponding to each frame image in the N frame image;

   A jitter amplitude value of each of the N frames of images is calculated based on the filtering process and the integrated processed signal.
3. The method of claim 2, wherein calculating the jitter amplitude value of each frame image in the N frame image according to the filtering process and the integrated processed signal comprises:

   A jitter amplitude value of each of the N frames of images is generated based on a difference between a maximum value and a minimum value of the filter processed and integrated processed signals.
4. The method of claim 2, wherein calculating the jitter amplitude value of each frame image in the N frame image according to the filtering process and the integrated processed signal comprises:

   Calculating the jitter amplitude value F _i of each of the N frames of images according to the following formula:

   

   Where i represents the frame number, ₁ ≤ _i ≤ N, η _i represents the number of sampling points t _{i of} the ith frame _{, j} represents the jth sampling time of the ith frame, and x _{i, j} represents the _{jth of} the ith frame Sample point signal value.
5. The method of claim 2, wherein calculating the jitter amplitude value of each frame image in the N frame image according to the filtering process and the integrated processed signal comprises:

   The sequence H _{i is} generated according to the filter processing and the integrated processed signal:

   

   Any calculation of characteristic parameters of the sequence H _i, the characteristic values of the following parameters: the mean absolute value of the sequence H _i, H _i the sequence of the absolute value and then average the results of all the elements, the said standard deviation of the sequence H _i, H _i is the variance of the sequence;

   Using the feature parameter as a jitter amplitude value of each frame image in the N frame image;

   among them,

   

   i represents the frame number, 1 ≤ _i ≤ N, η _i represents the number of sampling points of the ith frame, t _{i, j} represents the jth sampling time of the ith frame, and x _{i, j} represents the _{jth of} the ith frame Sample point signal value.
6. The method of claim 2, wherein calculating the jitter amplitude value of each frame image in the N frame image according to the filtering process and the integrated processed signal comprises:

   The sequence A _{i is} generated according to the filter processing and the integrated processed signal:

   

   Searching for the peaks and troughs of the sequence A _i , generating a jitter amplitude value of each frame image in the N frame image according to a difference between the peak point and the adjacent valley point;

   among them,

   

   x _i,j ′=x _i,j -x _i,1

   Where i represents the frame number, 1 ≤ _i ≤ N, η _i represents the number of sampling points of the ith frame, t _{i, j} represents the jth sampling time of the ith frame, and x _{i, j} represents the ith frame j sample point signal values.
7. The method according to any one of claims 1 to 6, wherein said image is based on said N frames The jitter amplitude value of each frame image in the selected K frame target image includes:

   The jitter amplitude values of each of the N frames of images are sorted, and the first K corresponding K frame images are taken as the K frame target images in ascending order.
8. The method according to any one of claims 1 to 7, wherein calculating the mutual positional offset of the K-frame target image comprises:

   Calculating a mutual positional offset of the K frame target image according to an output signal of the at least one inertial sensor corresponding to each frame image in the N frame image; or

   Feature points are extracted from the K-frame target image using an image detection algorithm, and mutual positional offsets of the feature points in the K-frame target image are calculated.
9. The method of any of claims 1 to 8, wherein the at least one inertial sensor comprises at least one of an angular velocity sensor and an acceleration sensor.
10. The method according to any one of claims 1 to 9, wherein before the acquiring the output signal of the at least one inertial sensor corresponding to each frame image in the N frame image, the method further includes:

    Detecting an output signal of at least one inertial sensor within a preset period of time before the start of exposure corresponding to each frame image of the N frames of images;

    Generating, according to an output signal of at least one inertial sensor in a preset period before the start of exposure corresponding to each frame image in the N frame image, a jitter prediction signal of each frame image in the N frame image;

    And an exposure start time of each of the N frames of images is controlled according to a jitter prediction signal of each of the N frames of images and an exposure duration of each of the N frames of images.
11. The method according to claim 10, wherein said generating said N frame based on an output signal of at least one inertial sensor within a preset period of time before exposure start corresponding to each frame image of said N frame image A jitter prediction signal for each frame of the image, including:

    And an output signal of at least one inertial sensor, a weight of each frequency component, and an exposure of each frame image in the N frame image according to a preset time period before exposure start corresponding to each frame image in the N frame image The duration generates a jitter prediction signal for each of the N frames of images, and the weights of the respective frequency components are used to estimate the frequency components The extent to which the output signal of the sensor affects imaging.
12. The method according to claim 10 or 11, wherein said exposure time control signal according to a jitter prediction signal of each frame image of said N frame image and each frame image of said N frame image The exposure start time of each frame image in the N frame image includes:

    Adjusting an exposure start time of each of the N frames of images according to a dither prediction signal of each of the N frames of images and an exposure duration of each of the N frames of images to cause The jitter prediction signal of each of the N frames of images is minimized by a difference between a maximum value and a minimum value within an exposure time period of each of the N frame images.
13. An anti-shake photographing device, comprising:

    An acquiring unit, configured to acquire an output signal of at least one inertial sensor corresponding to each frame image in the N frame image, N≥2;

    a jitter amplitude value calculation unit, configured to calculate a jitter amplitude value of each frame image in the N frame image according to an output signal of the at least one inertial sensor corresponding to each frame image in the N frame image;

    a target image selecting unit, configured to select a K frame target image according to a jitter amplitude value of each frame image in the N frame image, 2≤K≤N;

    a mutual position offset calculating unit, configured to calculate a mutual position offset of the K frame target image;

    And a final image generating unit configured to align the K frame target image according to mutual positional offset of the K frame target image and synthesize the aligned K frame target image to generate a final image.
14. The apparatus according to claim 13, wherein said jitter amplitude value calculation unit comprises:

    a filtering processing module, configured to perform filtering processing on an output signal of the at least one inertial sensor corresponding to each frame image in the N frame image;

    An integration processing module, configured to perform integral processing on an output signal of the filter processing module;

    a calculation module, configured to calculate the N according to an output signal of the integration processing module The jitter amplitude value of each frame image in the frame image.
15. The apparatus of claim 14, wherein the calculating module is specifically configured to:

    Generating a jitter amplitude value of each frame image in the N frame image according to a difference between a maximum value and a minimum value of an output signal of the integration processing module.
16. The apparatus of claim 14, wherein the calculating module is specifically configured to:

    Calculating the jitter amplitude value F _i of each of the N frames of images according to the following formula:

    

    Where i represents the frame number, 1 ≤ _i ≤ N, η _i represents the number of sampling points of the ith frame, t _{i, j} represents the jth sampling time of the ith frame, and x _{i, j} represents the ith frame j sample point signal values.
17. The apparatus of claim 14, wherein the calculating module is specifically configured to:

    Generating a sequence H _i according to an output signal of the integration processing module:

    

    Any calculation of characteristic parameters of the sequence H _i, the characteristic values of the following parameters: the mean absolute value of the sequence H _i, H _i the sequence of the absolute value and then average the results of all the elements, the said standard deviation of the sequence H _i, H _i is the variance of the sequence;

    Using the feature parameter as a jitter amplitude value of each frame image in the N frame image;

    among them,

    

    i represents the frame number, 1 ≤ _i ≤ N, η _i represents the number of sampling points of the ith frame, t _{i, j} represents the jth sampling time of the ith frame, and x _{i, j} represents the _{jth of} the ith frame Sample point signal value.
18. The apparatus of claim 14, wherein the calculating module is specifically configured to:

    Generating a sequence A _i according to an output signal of the integration processing module:

    

    The peaks and troughs of the search sequence A _i are generated based on the difference between the peak point and the adjacent valley point to generate a jitter amplitude value for each of the N frames of images.

    among them,

    

    x _i,j ′=x _i,j -x _i,1

    Where i represents the frame number, 1≤i≤N, 2≤k≤j, η _i represents the number of sampling points of the ith frame, and t _i,j represents the jth sampling time of the ith frame, x _i,j Indicates the value of the jth sample point signal of the ith frame.
19. The apparatus according to any one of claims 13 to 18, wherein the target image selecting unit is specifically configured to:

    The jitter amplitude values of each of the N frames of images are sorted, and the first K corresponding K frame images are taken as the K frame target images in ascending order.
20. The device according to any one of claims 13 to 19, wherein the mutual position offset calculation unit is specifically configured to:

    Calculating a mutual positional offset of the K frame target image according to an output signal of the at least one inertial sensor corresponding to each frame image in the N frame image; or

    Feature points are extracted from the K-frame target image using an image detection algorithm, and mutual positional offsets of the feature points in the K-frame target image are calculated.
21. A device according to any one of claims 13 to 20, wherein said at least one inertial sensor comprises at least one of an angular velocity sensor and an acceleration sensor.
22. The apparatus according to any one of claims 13 to 21, characterized in that the apparatus further comprises an exposure start time control unit for controlling an exposure start time of each of the N frames of images.
23. The apparatus according to claim 22, wherein said exposure start time control unit comprises:

    a detecting module, configured to detect an output signal of the at least one inertial sensor within a preset time period before the start of exposure corresponding to each frame image of the N frame image;

    Generating a module for each of the N frames of images detected by the detection module An output signal of at least one inertial sensor within a preset period of time before the start of exposure corresponding to one frame image generates a jitter prediction signal of each of the N frames of images;

    a control module, configured to control each of the N frame images according to a jitter prediction signal of each frame image in the N frame image generated by the generation module and an exposure duration of each frame image in the N frame image The exposure start time of the frame image.
24. The device according to claim 23, wherein the generating module is specifically configured to:

    And an output signal of at least one inertial sensor, a weight of each frequency component, and an exposure of each frame image in the N frame image according to a preset time period before exposure start corresponding to each frame image in the N frame image The duration generates a jitter prediction signal for each of the N frames of images, and the weights of the respective frequency components are used to evaluate the degree of influence of the output signals of the inertial sensors of the respective frequency components on the imaging.
25. The device according to claim 23 or 24, wherein the control module is specifically configured to:

    Adjusting an exposure start time of each of the N frames of images according to a dither prediction signal of each of the N frames of images and an exposure duration of each of the N frames of images to cause The jitter prediction signal of each of the N frames of images is minimized by a difference between a maximum value and a minimum value within an exposure time period of each of the N frame images.
26. A camera device, comprising:

    At least one inertial sensor for detecting jitter of the camera device corresponding to each frame image in the N frame image, N≥2;

    a processor, configured to calculate a jitter amplitude value of each frame image in the N frame image according to an output signal of the at least one inertial sensor; and select K according to a jitter amplitude value of each frame image in the N frame image a frame target image, 2≤K≤N; calculating a mutual positional offset of the K-frame target image; aligning the K-frame target image according to a mutual positional shift of the K-frame target image and synthesizing the aligned K Frame the target image to generate the final image.
27. The photographic apparatus according to claim 26, wherein said calculating each frame of said N frame image based on an output signal of said at least one inertial sensor The jitter amplitude value of the image includes:

    Performing filtering processing and integration processing on an output signal of at least one inertial sensor corresponding to each frame image in the N frame image;

    A jitter amplitude value of each of the N frames of images is calculated based on the filtering process and the integrated processed signal.
28. The photographic apparatus according to claim 27, wherein said calculating a jitter amplitude value of each frame image in said N frame image based on the filtering process and the integrated processed signal comprises:

    A jitter amplitude value of each of the N frames of images is generated based on a difference between a maximum value and a minimum value of the filter processed and integrated processed signals.
29. The photographic apparatus according to claim 27, wherein said calculating a jitter amplitude value of each frame image in said N frame image based on the filtering process and the integrated processed signal comprises:

    Calculating the jitter amplitude value F _i of each of the N frames of images according to the following formula:

    

    Where i represents the frame number, 1 ≤ _i ≤ N, η _i represents the number of sampling points of the ith frame, t _{i, j} represents the jth sampling time of the ith frame, and x _{i, j} represents the ith frame j sample point signal values.
30. The photographic apparatus according to claim 27, wherein said calculating a jitter amplitude value of each frame image in said N frame image based on the filtering process and the integrated processed signal comprises:

    The sequence H _{i is} generated according to the filter processing and the integrated processed signal:

    

    Any calculation of characteristic parameters of the sequence H _i, the characteristic values of the following parameters: the mean absolute value of the sequence H _i, H _i the sequence of the absolute value and then average the results of all the elements, the said standard deviation of the sequence H _i, H _i is the variance of the sequence;

    Using the feature parameter as the jitter amplitude of each frame image in the N frame image value;

    among them,

    

    Where i represents the frame number, 1 ≤ _i ≤ N, η _i represents the number of sampling points of the ith frame, t _{i, j} represents the jth sampling time of the ith frame, and x _{i, j} represents the ith frame j sample point signal values.
31. The photographic apparatus according to claim 27, wherein said processor calculates a jitter amplitude value of each frame image in said N frame image based on the filtering process and the integrated processed signal, comprising:

    The sequence A _{i is} generated according to the filter processing and the integrated processed signal:

    

    The peaks and troughs of the search sequence A _i are generated, and the jitter amplitude values of each of the N frames of images are generated based on the difference between the peak points and the adjacent valley points.

    among them,

    

    x _i,j ′=x _i,j -x _i,1

    Where i represents the frame number, 1≤i≤N, 2≤k≤j, η _i represents the number of sampling points of the ith frame, and t _i,j represents the jth sampling time of the ith frame, x _i,j Indicates the value of the jth sample point signal of the ith frame.
32. The photographic apparatus according to any one of claims 26 to 31, wherein the selecting a K-frame target image based on a jitter amplitude value of each frame image in the N-frame image comprises:

    The jitter amplitude values of each of the N frames of images are sorted, and the first K corresponding K frame images are taken as the K frame target images in ascending order.
33. The photographic apparatus according to any one of claims 26 to 32, wherein said calculating a mutual positional shift of said K-frame target image comprises:

    Calculating a mutual positional offset of the K frame target image according to an output signal of the at least one inertial sensor corresponding to each frame image in the N frame image; or

    Feature points are extracted from the K-frame target image using an image detection algorithm, and mutual positional offsets of the feature points in the K-frame target image are calculated.
34. A photographic apparatus according to any one of claims 26 to 33, wherein said at least one inertial sensor comprises at least one of an angular velocity sensor and an acceleration sensor.
35. The photographic device according to any one of claims 26 to 34, wherein the processor is further configured to: acquire at least a preset period of time before the start of exposure corresponding to each frame image of the N frames of images An output signal of the inertial sensor; generating an image of each of the N frames of images according to an output signal of the at least one inertial sensor within a preset period of time before the start of exposure corresponding to each frame image of the N frames of images a jitter prediction signal; controlling exposure start of each frame image in the N frame image according to a jitter prediction signal of each frame image in the N frame image and an exposure duration of each frame image in the N frame image time.
36. The photographic apparatus according to claim 35, wherein said generating said N based on an output signal of at least one inertial sensor within a preset period of time before exposure start corresponding to each frame image of said N frame image The jitter prediction signal of each frame image in the frame image includes:

    And an output signal of at least one inertial sensor, a weight of each frequency component, and an exposure of each frame image in the N frame image according to a preset time period before exposure start corresponding to each frame image in the N frame image The duration generates a jitter prediction signal for each of the N frames of images, and the weights of the respective frequency components are used to evaluate the degree of influence of the output signals of the inertial sensors of the respective frequency components on the imaging.
37. The photographing apparatus according to claim 35 or 36, wherein said exposure time length control signal according to each of said N frames of images and exposure time control of each of said N frames of images The exposure start time of each frame image in the N frame image includes:

    Adjusting an exposure start time of each of the N frames of images according to a dither prediction signal of each of the N frames of images and an exposure duration of each of the N frames of images to cause a maximum value and a minimum of a jitter prediction signal of each frame image in the N frame image within an exposure time period of each frame image in the N frame image The difference in values is minimal.

Images