CN116546324A

CN116546324A - Video anti-shake method and device

Info

Publication number: CN116546324A
Application number: CN202310565450.4A
Authority: CN
Inventors: 胡浩棋; 胡庆枫
Original assignee: Vivo Mobile Communication Co Ltd
Current assignee: Vivo Mobile Communication Co Ltd
Priority date: 2023-05-18
Filing date: 2023-05-18
Publication date: 2023-08-04

Abstract

The application discloses a video anti-shake method and a device thereof, belonging to the technical field of anti-shake. The method comprises the following steps: responding to a first input of a user, displaying a preview interface, wherein the preview interface comprises an anti-shake control, and the anti-shake control is used for adjusting anti-shake parameters; responding to the second input of the user to the preview interface, and displaying a video recording interface; acquiring at least two frames of images based on the first anti-shake parameters; performing anti-shake processing on at least two frames of images based on the second anti-shake parameters to obtain a target video; the first anti-shake parameter and the second anti-shake parameter are determined based on the anti-shake control, the first anti-shake parameter indicates an optical anti-shake parameter, the second anti-shake parameter indicates an electronic anti-shake parameter, and different second inputs correspond to different first anti-shake parameter and second anti-shake parameter.

Description

Video anti-shake method and device

Technical Field

The application belongs to the technical field of anti-shake, and particularly relates to a video anti-shake method and a device thereof.

Background

Electronic devices with cameras basically have video shooting functions, and cameras are generally required to have anti-shake functions. Currently, anti-shake generally includes two modes, electronic anti-shake and optical anti-shake. The electronic anti-shake is realized by cutting the picture, and the mode can sacrifice part of the image view angle; the optical anti-shake can compensate external shake by adjusting the relative position between the lens and the sensor, and the problem of image blurring caused by external shake can be effectively solved.

Disclosure of Invention

The embodiment of the application aims to provide a video anti-shake method and a device thereof, which can flexibly adjust the anti-shake effect and can give consideration to the image quality in the anti-shake process.

In a first aspect, an embodiment of the present application provides a video anti-shake method, including:

responding to a first input of a user, displaying a preview interface, wherein the preview interface comprises an anti-shake control, and the anti-shake control is used for adjusting anti-shake parameters;

responding to the second input of the user to the preview interface, and displaying a video recording interface;

acquiring at least two frames of images based on the first anti-shake parameters;

performing anti-shake processing on at least two frames of images based on the second anti-shake parameters to obtain a target video;

the first anti-shake parameter and the second anti-shake parameter are determined based on the anti-shake control, the first anti-shake parameter indicates an optical anti-shake parameter, the second anti-shake parameter indicates an electronic anti-shake parameter, and different second inputs correspond to different first anti-shake parameter and second anti-shake parameter.

In a second aspect, an embodiment of the present application provides a video anti-shake apparatus, including:

the first display module is used for responding to the first input of the user, displaying a preview interface, wherein the preview interface comprises an anti-shake control, and the anti-shake control is used for adjusting anti-shake parameters;

The second display module is used for responding to the second input of the user to the preview interface and displaying a video recording interface;

the acquisition module is used for acquiring at least two frames of images based on the first anti-shake parameters;

the processing module is used for carrying out anti-shake processing on at least two frames of images based on the second anti-shake parameters to obtain a target video;

In a third aspect, embodiments of the present application provide an electronic device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the method as described in the first aspect.

In a fourth aspect, embodiments of the present application provide a readable storage medium having stored thereon a program or instructions which when executed by a processor implement the steps of the method according to the first aspect.

In a fifth aspect, embodiments of the present application provide a chip, where the chip includes a processor and a communication interface, where the communication interface is coupled to the processor, and where the processor is configured to execute a program or instructions to implement a method according to the first aspect.

In a sixth aspect, embodiments of the present application provide a computer program product stored in a storage medium, the program product being executable by at least one processor to implement the method according to the first aspect.

In the embodiment of the application, the anti-shake control for adjusting the anti-shake parameter can be displayed on the preview interface, and the user can perform different second inputs on the anti-shake control based on different video shooting scenes, so that different optical anti-shake parameters and different electronic anti-shake parameters are determined. Therefore, the relation between the anti-shake intensity and the image quality can be coordinated through different optical anti-shake parameters and electronic anti-shake parameters, the simplification of an adjustment mode caused by using fixed anti-shake parameters is avoided, the anti-shake effect is flexibly adjusted, the purpose of the image quality can be achieved in the anti-shake process, and the video recording requirement of a user can be met.

Drawings

Fig. 1 is a flow chart of a video anti-shake method according to an embodiment of the present application;

Fig. 2 is an interface schematic diagram in a video anti-shake method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an anti-shake control in a video anti-shake method according to an embodiment of the present disclosure;

FIG. 4 is another schematic diagram of an anti-shake control in the video anti-shake method according to the embodiment of the present application;

fig. 5 is a schematic structural diagram of a video anti-shake apparatus according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

fig. 7 is a schematic hardware structure of an electronic device according to an embodiment of the present application.

Detailed Description

Technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application are within the scope of the protection of the present application.

The terms first, second and the like in the description and in the claims, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged, as appropriate, such that embodiments of the present application may be implemented in sequences other than those illustrated or described herein, and that the objects identified by "first," "second," etc. are generally of a type and not limited to the number of objects, e.g., the first object may be one or more. Furthermore, in the description and claims, "and/or" means at least one of the connected objects, and the character "/", generally means that the associated object is an "or" relationship.

The video anti-shake method provided by the embodiment of the application is described in detail below by means of specific embodiments and application scenes thereof with reference to the accompanying drawings.

It can be understood that, currently, in the video recording process, fixed electronic anti-shake parameters and optical anti-shake parameters are usually set to achieve the purpose of video anti-shake. That is, the viewing angle cut-out ratio is generally fixed in different anti-shake modes at present, and the switching of the viewing angle cut-out ratio is performed only by limited mode selection. At present, the movable angle of the lens in the optical anti-shake process is also fixed, and the size of the optical anti-shake angle cannot be changed in the use process. However, this fixed and single adjustment is often difficult to meet the video recording needs of the user.

For electronic anti-shake, considering that users have different emphasis on the view angles and the anti-shake effect under different scenes, for example, more view angles can be sacrificed to achieve better anti-shake effect when recording a bamboo sea; the angle of view is more important when shooting close-up tall scenes. In addition, the stability of the handheld video of different users is different, and for the users with stable handholding, the cutting proportion is not required to be too high, the viewing angle can be enlarged by reducing the cutting proportion of the viewing angle, otherwise, for the users with shaking handholding, the shooting stability in the recording process can be improved by accepting a smaller viewing angle.

For optical anti-shake, from the perspective of an actual user scene, when the user holds still to photograph, the shake is smaller, so that the user does not need to provide too large anti-shake intensity, and when the user runs or jumps, the shake intensity is increased, and a large optical anti-shake angle is needed for high-intensity anti-shake. In addition, for the user with stable hands, an excessive optical anti-shake angle is not needed, the image quality can be improved by reducing the optical anti-shake angle, and for the user with larger hand-held shake, the image stability is improved by needing a larger optical anti-shake angle, the edge image quality is properly sacrificed, and the optimal user experience is achieved.

Based on this, in order to solve the above technical problems, the embodiments of the present application provide a video anti-shake method that can flexibly adjust the anti-shake effect and can consider the image quality in the anti-shake process. Fig. 1 is a flowchart of a video anti-shake method according to an embodiment of the present application. The video anti-shake method may include:

step 101, in response to a first input of a user, displaying a preview interface, wherein the preview interface comprises an anti-shake control, and the anti-shake control is used for adjusting anti-shake parameters.

In step 101, as shown in fig. 2, when a user wants to record video using the electronic device, the electronic device may receive a first input from the user, and display a preview interface 201 in response to the first input, where the preview interface 201 may include an anti-shake control 202 for adjusting an anti-shake parameter. Wherein the first input may be: the click input of the user, or the voice command input by the user, or the specific gesture input by the user, may be specifically determined according to the actual use requirement, which is not limited in the embodiment of the present application. The specific gesture in the embodiment of the application may be any one of a single-click gesture, a sliding gesture, a dragging gesture, a pressure recognition gesture, a long-press gesture, an area change gesture, a double-press gesture and a double-click gesture; the click input in the embodiment of the application may be single click input, double click input, or any number of click inputs, and may also be long press input or short press input.

The anti-shake parameters have an influence on both the anti-shake intensity and the image quality. The anti-shake parameters may include a first anti-shake parameter and a second anti-shake parameter, where the first anti-shake parameter may indicate an optical anti-shake parameter, such as an optical anti-shake angle, and may be an angle at which a lens is movable. The second anti-shake parameter may indicate an electronic anti-shake parameter, such as a viewing angle cut-out ratio.

It can be appreciated that electronic anti-shake, i.e., anti-shake techniques that achieve video stabilization by sacrificing image field angle. Under the same condition, the larger the cutting proportion of the angle of view is, the larger the anti-shake compensation range is, so that the image stabilizing effect of different degrees can be realized by setting different cutting proportions of the angle of view. In other words, the anti-shake strength is affected by the viewing angle cut-out ratio, and the stronger the anti-shake strength is, the smaller the viewing angle is.

The optical anti-shake is to compensate the image shake caused by the external shake by adjusting the relative position between the lens and the sensor. The problem that dark angle appears at picture edge, and the definition is relatively poor can be brought to too big optical anti-shake angle, in other words, the optical anti-shake angle is bigger just can compensate the shake of external high strength more, but can bring the loss of image quality simultaneously, and optical anti-shake angle is little, and the image quality influence is little but anti-shake intensity weakens.

Therefore, the anti-shake effect and the image quality both affect the final video recording effect, which are important experience dimensions of the user in video recording, and the anti-shake effect and the image quality are in a trade-off relationship. Based on the relation between the anti-shake effect and the image quality can be weighed by the user through the anti-shake control on the preview interface.

Step 102, in response to a second input from the user to the preview interface, displaying a video recording interface.

In step 102, the different second inputs may correspond to different first and second anti-shake parameters. In other words, a second input of the user to the preview interface may be received, and the video recording interface may be displayed in response to the second input, where at the same time, the electronic device may determine, based on the second input, a first anti-shake parameter and a second anti-shake parameter required by the user, and record the video according to the first anti-shake parameter and the second anti-shake parameter.

Wherein the second input may be: the click input of the user on the preview interface, or the voice command input by the user, or the specific gesture input by the user, may be specifically determined according to the actual use requirement, which is not limited in the embodiment of the present application. The specific gesture in the embodiment of the application may be any one of a single-click gesture, a sliding gesture, a dragging gesture, a pressure recognition gesture, a long-press gesture, an area change gesture, a double-press gesture and a double-click gesture; the click input in the embodiment of the application may be single click input, double click input, or any number of click inputs, and may also be long press input or short press input.

And step 103, acquiring at least two frames of images based on the first anti-shake parameters.

In step 103, as described above, the first anti-shake parameter may indicate an optical anti-shake parameter, and at least two frames of images may be acquired using a specific value of the optical anti-shake angle indicated by the first anti-shake parameter.

And 104, performing anti-shake processing on at least two frames of images based on the second anti-shake parameters to obtain a target video.

In step 104, as described above, the second anti-shake parameter may indicate an electronic anti-shake parameter, that is, a specific value of the cutting ratio of the field angle indicated by the second anti-shake parameter may be used to cut at least two acquired frames of images, so as to obtain the target video after anti-shake processing.

Considering that the user has different emphasis on the view angle and the anti-shake effect under different scenes, for example, more view angles can be sacrificed to achieve better anti-shake effect when recording a bamboo sea; the angle of view is more important when shooting close-up tall scenes. In addition, the stability of the handheld video of different users is different, and for the users with stable handholding, the cutting proportion is not required to be too high, the viewing angle can be enlarged by reducing the cutting proportion of the viewing angle, otherwise, for the users with shaking handholding, the shooting stability in the recording process can be improved by accepting a smaller viewing angle.

In the prior art, the movable angle of the lens in the optical anti-shake process is also fixed, and the size of the optical anti-shake angle cannot be changed in the use process, so that inconvenience is brought to users. From the actual user scene, when the user holds still to shoot, the shake is smaller and the shake strength is not required to be too large, when the user runs or jumps, the shake strength is increased, and a large optical shake prevention angle is required to perform high-strength shake prevention. In addition, for the user with stable hands, an excessive optical anti-shake angle is not needed, the image quality can be improved by reducing the optical anti-shake angle, and for the user with larger hand-held shake, the image stability is improved by needing a larger optical anti-shake angle, and the edge image quality is properly sacrificed.

Therefore, the existing video anti-shake mode is single, and the video anti-shake mode is difficult to meet the wider video recording scene requirements of users. Based on this, in the embodiment of the application, the video anti-shake method can display the anti-shake control for adjusting the anti-shake parameter on the preview interface, and the user can perform different second inputs on the anti-shake control based on different video shooting scenes, so as to determine different optical anti-shake parameters and electronic anti-shake parameters. Therefore, the relation between the anti-shake intensity and the image quality can be coordinated through different optical anti-shake parameters and electronic anti-shake parameters, the simplification of an adjustment mode caused by using fixed anti-shake parameters is avoided, the anti-shake effect is flexibly adjusted, the effect of the image quality can be considered in the anti-shake process, and the video recording requirement of a user can be met.

In some embodiments, the anti-shake control comprises a control body and an identifier located on the control body; the second input includes a first sub-input to the identity;

the step 102 may include the following steps:

determining a first anti-shake parameter and a second anti-shake parameter corresponding to the first sub-input in response to the first sub-input of the identifier by the user, wherein the first anti-shake parameter and the second anti-shake parameter have an association relationship with the input parameter of the first sub-input;

and displaying a video recording interface.

In this embodiment, the first anti-shake parameter and the second anti-shake parameter corresponding to the first sub-input may be determined in response to the first sub-input of the identifier by the user. The first anti-shake parameter and the second anti-shake parameter may have an association relationship with the input parameter of the first sub-input.

For example, as shown in FIG. 3, an anti-shake control may include a control body 301 and a logo 302 located on the control body 301, the logo 302 being operable to move in response to an input location of a first sub-input. The control body 301 may be a sliding bar with scales, different scales may correspond to different anti-shake parameters, the left side may indicate that the image display effect is better, that is, the image quality is optimal, at this time, the viewing angle cutting proportion and the optical anti-shake angle are minimum, and the right end may indicate that the anti-shake effect is better, at this time, the viewing angle cutting proportion and the optical anti-shake angle are maximum.

In other words, different anti-shake parameters may be associated with different positions of the control body 301, and target position information of the identifier 302 on the control body 301 may be determined in response to the first sub-input, so as to determine a first anti-shake parameter and a second anti-shake parameter associated with the target position information. Therefore, the mark can be manually adjusted to the target position information on the control body according to the selection of a user so as to determine the proper first anti-shake parameter and second anti-shake parameter, the anti-shake effect is flexibly adjusted, the purpose of image quality can be achieved in the anti-shake process, and the user requirement can be met.

For example, the shake condition of the current electronic device may be obtained in response to the first sub-input, so as to determine the required shake strength, and further determine the corresponding first shake parameter and the second shake parameter based on the shake strength. The specific manner may be flexibly set, and is not particularly limited herein.

After the first anti-shake parameter and the second anti-shake parameter corresponding to the first sub-input are determined, the video recording interface can be automatically displayed without the need of operation of a user, and video recording can be automatically performed according to the first anti-shake parameter and the second anti-shake parameter.

Therefore, the first sub-input of the user to the identifier can be responded, and the first anti-shake parameter and the second anti-shake parameter required by the user can be determined, so that video recording can be performed according to the first anti-shake parameter and the second anti-shake parameter, the anti-shake strength can be flexibly adjusted, the effect of image quality can be achieved in the anti-shake process, and the video recording requirement of the user can be met.

In some embodiments, the second input includes a second sub-input, and the video anti-shake method may further include the following steps before the video recording interface is displayed:

receiving a second sub-input of the user to the preview interface;

the displaying the video recording interface may further include the following steps:

in response to the second sub-input, a video recording interface is displayed.

In this embodiment, after receiving the first sub-input of the identifier by the user, only the first sub-input may be responded, and the first anti-shake parameter and the second anti-shake parameter corresponding to the first sub-input may be determined, and at this time, the video recording interface may not be displayed, and the user may continue to adjust the anti-shake parameter until the first anti-shake parameter and the second anti-shake parameter that best meet the requirement of the user are determined. A second sub-input to the preview interface by the user may then be received and the video recording interface displayed in response to the second sub-input.

Therefore, the user can freely adjust the anti-shake parameters based on the anti-shake control until the first anti-shake parameter and the second anti-shake parameter which best meet the requirements of the user are determined, and then the video recording interface can be displayed in response to the second sub-input of the user to the preview interface so that video recording can be performed according to the first anti-shake parameter and the second anti-shake parameter, and the video recording requirements of the user are met.

In some embodiments, the determining the first anti-shake parameter and the second anti-shake parameter corresponding to the first sub-input may include the following steps:

acquiring gyroscope sensor data corresponding to N frames of preview images in a first time period, wherein N is an integer greater than 1;

according to the gyroscope sensor data corresponding to the N frames of preview images, N rotation matrixes corresponding to the N frames of preview images are determined;

determining anti-shake intensity according to the N rotation matrixes;

and determining a first anti-shake parameter and a second anti-shake parameter according to the anti-shake intensity.

In this embodiment, considering that it is difficult for the user to grasp a suitable anti-shake effect, the anti-shake parameter may be adaptively adjusted to a suitable anti-shake parameter according to the shake condition of the current electronic device. For example, the gyroscope sensor data of a preview image of a certain frame number can be used for video preview, an inter-frame rotation matrix is calculated, the current shaking intensity is estimated, and the required shaking intensity is determined, so that the appropriate first shaking prevention parameter and second shaking prevention parameter are selected.

For example, as shown in FIG. 4, an adaptive anti-shake approach may be turned on in response to pulling down on a first sub-input of the logo. At this time, gyroscope sensor data corresponding to N frames of preview images in the first period of time may be acquired. The first period may be a preset period, or a period determined according to the input parameter of the first sub-input, which is not specifically limited herein.

N rotation matrixes corresponding to the N frames of preview images can be determined according to the gyroscope sensor data corresponding to the N frames of preview images. The rotation matrix can be calculated by adopting an Euler angle mode or a quaternion mode.

In one example, the gyro sensor data may include angular velocity ω (t) = (ω) _x ,ω _y ,ω _z ) The shake angle (θ (n)) of each preview image can be calculated from this _x ,θ(n) _y ,θ(n) _z ). Wherein the jitter angle (θ (n)) _x ,θ(n) _y ,θ(n) _z ) The calculation formula of (2) can be shown as formula (1):

where Δt may represent the sampling interval of the gyroscope and k may represent the number of times the gyroscope is sampled between frames.

Can be based on the dithering angle (θ (n)) _x ,θ(n) _y ,θ(n) _z ) The rotation matrix of each preview image is represented by euler angles, wherein the calculation formula of the rotation matrix R can be as shown in formula (2):

In another example, the gyro sensor data may further include a unit vector of a rotation axis, and an angle of rotation around the rotation axis, so that there is one rotation axis u, and an expression of a rotation quaternion of a rotation angle around the u axis is as shown in formula (3):

where q may be a unit quaternion and u may represent a unit vector of the rotation axis.

From this, the quaternion q= (Q) of each preview image can be obtained ₀ ,q ₁ ,q ₂ ,q ₃ ) And according to the quaternion, calculating to obtain a rotation matrix of each preview image, wherein a calculation formula of the rotation matrix R can be shown as a formula (4):

the anti-shake intensity may be determined from the N rotation matrices. For example, after N rotation matrices of the N frame preview images are calculated, the jitter condition of the current electronic device may be determined according to the conditions of the rotation matrices. The change of the rotation matrix in the same direction in a very short time can be considered as reasonable user action, and the direction difference is abnormal jitter. Therefore, it can be considered that the higher the similarity of the inter-frame rotation matrix, the lower the jitter frequency, the smaller the jitter intensity, and the required anti-jitter intensity, whereas the lower the similarity, the greater the jitter intensity, and the greater the required anti-jitter intensity.

The first anti-shake parameter and the second anti-shake parameter may be determined according to the anti-shake intensity. For example, a corresponding relationship between the anti-shake intensity and the anti-shake parameter may be preset, and the first anti-shake parameter and the second anti-shake parameter are matched according to the determined anti-shake intensity. The first anti-shake parameter and the second anti-shake parameter may be calculated based on the anti-shake intensity according to a maximum or minimum anti-shake parameter that can be supported by the electronic device. The present invention is not particularly limited herein.

Therefore, the gyroscope sensor data of the preview image with a certain frame number can be utilized to estimate the current shaking intensity, and the required shaking intensity is determined, so that more accurate first shaking prevention parameters and second shaking prevention parameters are obtained, and the shaking prevention effect is effectively ensured.

In some embodiments, determining the anti-shake intensity from the N rotation matrices includes:

based on the rotation matrixes corresponding to the adjacent two frames of preview images, calculating to obtain N-1 rotation matrix similarity;

determining average similarity according to the N-1 rotation matrix similarity;

and determining the anti-shake strength according to the average similarity.

In this embodiment, as described above, the anti-shake strength is related to the similarity of the rotation matrices between frames, based on which the similarity of the rotation matrices corresponding to the two adjacent frames of preview images can be determined according to the rotation matrices of the two adjacent frames of preview images, so as to obtain N-1 rotation matrix similarities.

Illustratively, taking two adjacent frames of preview images as an n-1 frame image and an n frame image as examples, the matrix similarity ρ is rotated _(n-1) The calculation formula of (2) can be as shown in formula (5):

wherein R '(n-1) is a one-dimensional vector flattened by a rotation matrix R (n-1) of an n-1 th frame image, and R' (n) is a one-dimensional vector flattened by a rotation matrix R (n) of an n-th frame image. The calculated similarity in this embodiment is cosine similarity.

The average similarity can be determined according to N-1 rotation matrix similarityFor example, average similarity +.>May be (ρ) ₁ +ρ ₂ +…+ρ _n-2 +ρ _n-1 )/(n-1)。

The anti-shake intensity ζ may be determined based on the average similarity. Illustratively, the calculation formula of the anti-shake intensity ζ may be as shown in formula (6):

wherein, xi is the anti-shake intensity,is the average similarity.

In this way, the average similarity can be determined through the N-1 rotation matrix similarity of the preview images of two adjacent frames, so that the accurate anti-shake intensity can be obtained through calculation, the more accurate first anti-shake parameter and the second anti-shake parameter can be obtained, and the anti-shake effect is effectively ensured.

In some embodiments, determining the average similarity from the N-1 rotation matrix similarities comprises:

and carrying out weighted average on the N-1 rotation matrix similarity to obtain average similarity.

In this embodiment, the weights corresponding to the similarities of each rotation matrix may be identical or non-identical, for example, a weight value may be set for each rotation matrix similarity in advance according to an empirical value, and then the average similarity is obtained by weighted average calculation based on N-1 rotation matrix similarities and the weight values corresponding to the rotation matrix similarities.

For example, when the N-1 rotation matrix similarities are weighted and averaged, the fact that the handheld state of the handheld electronic device of the user is influenced by time can be considered, and the jitter vector behind the time can reflect the real jitter trend more, so that the rotation matrix similarities of two adjacent frames of preview images in the N frames of preview images can be weighted linearly. For example, time information of each preview image in the N frames of preview images may be acquired, and weights corresponding to N-1 rotation matrix similarities of adjacent two frames of preview images may be determined according to the time information of each preview image. It can be understood that the more the time information is, the heavier the weight corresponding to the rotation matrix similarity of the two adjacent frames of preview images is, and the weight change can be in linear distribution. The average similarity may be determined based on the N-1 rotation matrix similarities and their respective weights. Average similarity The calculation formula of (2) can be as shown in formula (7):

wherein, the liquid crystal display device comprises a liquid crystal display device,to average similarity ρ ₁ Corresponding to the 1 st frame image and the 2 nd frame imageRotation matrix similarity of ρ ₂ For the rotation matrix similarity, ρ, of the 2 nd frame image and the 3 rd frame image _n-2 For the similarity of the rotation matrix corresponding to the n-2 frame image and the n-1 frame image, ρ _n-1 The rotation matrix similarity between the n-1 frame image and the n frame image is obtained.

In this way, the influence of external factors such as time on the jitter condition can be considered, weighted average is carried out on the N-1 rotation matrix similarity, and the accuracy of the average similarity is further ensured, so that more accurate anti-jitter intensity is obtained, more accurate first anti-jitter parameters and second anti-jitter parameters are matched, and the anti-jitter effect is further ensured.

In some embodiments, determining the first anti-shake parameter and the second anti-shake parameter from the anti-shake intensity includes:

acquiring a maximum first anti-shake parameter, a minimum first anti-shake parameter, a maximum second anti-shake parameter and a minimum second anti-shake parameter of the electronic equipment;

determining a first anti-shake parameter according to the anti-shake intensity, the maximum first anti-shake parameter and the minimum first anti-shake parameter;

and determining a second anti-shake parameter according to the anti-shake intensity, the maximum second anti-shake parameter and the minimum second anti-shake parameter.

In this embodiment, the maximum first anti-shake parameter, the minimum first anti-shake parameter, the maximum second anti-shake parameter, and the minimum second anti-shake parameter of the electronic device, that is, the maximum optical anti-shake angle, the minimum optical anti-shake angle, the maximum viewing angle clipping ratio, and the minimum viewing angle clipping ratio that can be supported by the electronic device, may also be obtained.

The first anti-shake parameter may be determined according to the anti-shake intensity, the maximum first anti-shake parameter, and the minimum first anti-shake parameter. As noted above, the first anti-shake parameter is the optical anti-shake angle deg _ois Optical anti-shake angle deg _ois The calculation formula of (2) can be shown as formula (8):

deg _ois ＝ξ*Max _deg +(1-ξ)*Min _deg (8)

wherein, xi is anti-shake intensity, min _deg For a minimum optical anti-shake angle,Max _deg for maximum optical anti-shake angle, min _deg And Max _deg Is a super parameter.

The second anti-shake parameter may be determined according to the anti-shake intensity, the maximum second anti-shake parameter, and the minimum second anti-shake parameter. As described above, the second anti-shake parameter is exemplified by the angle of view clipping ratio (margin, margin w), and the calculation formula of the angle of view clipping ratio (margin, margin w) may be as shown in formula (9):

marginH＝marginW＝ξ*Max _margin +(1-ξ)*Min _margin (9)

wherein, xi is anti-shake intensity, max _margin For maximum angle of view cut ratio, min _margin Max for minimum viewing angle cut-out ratio _margin And Min _margin Is a super parameter.

Therefore, the first anti-shake parameter and the second anti-shake parameter can be obtained by calculating the anti-shake intensity, the maximum first anti-shake parameter, the minimum first anti-shake parameter, the maximum second anti-shake parameter and the minimum second anti-shake parameter which can be supported by the electronic equipment, the accuracy of the first anti-shake parameter and the second anti-shake parameter and the suitability of the electronic equipment are ensured, and the anti-shake effect is further ensured.

In some embodiments, the first sub-input is a sliding input, the first time period is determined based on input parameters of the first sub-input, the input parameters including at least one of: sliding distance, input time.

In this embodiment, the first sub-input may be a sliding input, and the first period of time may be determined based on a sliding distance of the sliding input. For example, as shown in fig. 4, after the mark is slid downwards for a certain distance, the mark returns to the control body at a fixed speed v, the longer the dragging distance, the longer the duration of time t for returning to the control body, and the preview image can be acquired in the process of returning to the control body. I.e. the longer the sliding distance of the sliding input, the longer the first time period. At this time, the more the number of frames n of the preview image can be obtained, the more the gyroscope sensor data can be obtained, and the higher the accuracy of the subsequent anti-shake intensity calculation can be.

The first time period may also be determined based on the input time of the first sub-input, e.g., the longer the input time, the longer the first time period, the shorter the input time, and the shorter the opposite first time period. The number of frames for obtaining the preview image can be controlled by adjusting the input time, so that the purpose of controlling the accuracy of the subsequent calculation of the anti-shake intensity is achieved.

The first time period may also be determined jointly based on the input time and the sliding distance of the first sub-input. The larger the sliding distance is, the longer the input time is, the longer the corresponding first time period is, the smaller the sliding distance is, and the shorter the input time is, the shorter the corresponding first time period is. For example, considering two dimensions of the input time and the sliding distance, corresponding weight values may be set for the input time and the sliding distance, respectively, and the duration of the first period may be calculated together by the respective weight values and the duration of the input time and the length of the sliding distance. Thus, the frame number of the obtained preview image can be controlled jointly by adjusting the input time and the sliding distance, so that the purpose of controlling the accuracy of the subsequent calculation of the anti-shake intensity is achieved.

Therefore, the duration of the first time period can be controlled based on the input parameters of the first sub-input, so that the anti-shake intensity precision control is realized, and the flexibility of video anti-shake is further improved.

According to the video anti-shake method provided by the embodiment of the application, the execution main body can be a video anti-shake device. In the embodiment of the application, a method for executing video anti-shake by using a video anti-shake device is taken as an example, and the video anti-shake device provided in the embodiment of the application is described.

As shown in fig. 5, a video anti-shake apparatus 500 provided in an embodiment of the present application may include:

a first display module 501, configured to respond to a first input of a user, and display a preview interface, where the preview interface includes an anti-shake control, and the anti-shake control is used to adjust an anti-shake parameter;

a second display module 502, configured to display a video recording interface in response to a second input of the preview interface by the user;

the acquisition module 503 is configured to acquire at least two frames of images based on the first anti-shake parameter;

the processing module 504 is configured to perform anti-shake processing on at least two frames of images based on the second anti-shake parameter to obtain a target video;

the second display module 502 may include:

the determining unit is used for responding to the first sub-input of the user for the identification, determining a first anti-shake parameter and a second anti-shake parameter corresponding to the first sub-input, wherein the first anti-shake parameter and the second anti-shake parameter have an association relation with the input parameter of the first sub-input;

and the display unit is used for displaying the video recording interface.

Therefore, the first sub-input of the user to the mark can be responded, and the first anti-shake parameter and the second anti-shake parameter required by the user can be determined, so that video recording can be performed according to the first anti-shake parameter and the second anti-shake parameter, the anti-shake effect can be flexibly adjusted, the purpose of taking the image quality into account in the anti-shake process can be achieved, and the user requirement can be met.

In some embodiments, the second input includes a second sub-input, and the video anti-shake apparatus 500 may further include:

the receiving module is used for receiving a second sub-input of the preview interface from the user;

the display unit may also be used to:

in response to the second sub-input, a video recording interface is displayed.

In some embodiments, the determining unit may further include:

the acquisition subunit is used for acquiring gyroscope sensor data corresponding to N frames of preview images in a first time period, wherein N is an integer greater than 1;

the first determining subunit is used for determining N rotation matrixes corresponding to the N frames of preview images according to the gyroscope sensor data corresponding to the N frames of preview images;

the second determining subunit is used for determining the anti-shake intensity according to the N rotation matrixes;

and the third determination subunit is used for determining the first anti-shake parameter and the second anti-shake parameter according to the anti-shake intensity.

In some embodiments, the second determination subunit may be further configured to:

determining average similarity according to the N-1 rotation matrix similarity;

and determining the anti-shake strength according to the average similarity.

In this way, the average similarity of N frames of preview images can be determined through the N-1 rotation matrix similarity of the adjacent two frames of preview images, so that accurate anti-shake intensity can be obtained through calculation, more accurate first anti-shake parameters and second anti-shake parameters can be obtained, and the anti-shake effect is effectively ensured.

In some embodiments, the third determination subunit may be further configured to:

The video anti-shake device in the embodiment of the application may be an electronic device, or may be a component in the electronic device, for example, an integrated circuit or a chip. The electronic device may be a terminal, or may be other devices than a terminal. By way of example, the electronic device may be a mobile phone, tablet computer, notebook computer, palm computer, vehicle-mounted electronic device, mobile internet appliance (Mobile Internet Device, MID), augmented reality (augmented reality, AR)/Virtual Reality (VR) device, robot, wearable device, ultra-mobile personal computer, UMPC, netbook or personal digital assistant (personal digital assistant, PDA), etc., but may also be a server, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (TV), teller machine or self-service machine, etc., and the embodiments of the present application are not limited in particular.

The video anti-shake device in the embodiment of the application may be a device with an operating system. The operating system may be an Android operating system, an ios operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.

The video anti-shake device provided in the embodiment of the present application can implement each process implemented by the embodiments of the methods of fig. 1 to fig. 4, and in order to avoid repetition, a detailed description is omitted here.

Optionally, as shown in fig. 6, the embodiment of the present application further provides an electronic device 600, including a processor 601 and a memory 602, where a program or an instruction capable of running on the processor 601 is stored in the memory 602, and the program or the instruction implements each step of the embodiment of the video anti-shake method when executed by the processor 601, and the steps can achieve the same technical effects, so that repetition is avoided, and no further description is given here.

The electronic device in the embodiment of the application includes the mobile electronic device and the non-mobile electronic device described above.

Fig. 7 is a schematic hardware structure of an electronic device implementing an embodiment of the present application.

The electronic device 700 includes, but is not limited to: radio frequency unit 701, network module 702, audio output unit 703, input unit 704, sensor 705, display unit 706, user input unit 707, interface unit 708, memory 709, and processor 710.

Those skilled in the art will appreciate that the electronic device 700 may also include a power source (e.g., a battery) for powering the various components, which may be logically connected to the processor 710 via a power management system so as to perform functions such as managing charge, discharge, and power consumption via the power management system. The electronic device structure shown in fig. 7 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than shown, or may combine certain components, or may be arranged in different components, which are not described in detail herein.

Wherein the display unit 706 may be configured to:

the processor 710 may be configured to:

In some embodiments, the anti-shake control comprises a control body and an identifier located on the control body, and the second input comprises a first sub-input to the identifier;

the processor 710 may also be configured to:

the display unit 706 may also be configured to:

and displaying a video recording interface.

In some embodiments, the second input comprises a second sub-input, and the user input unit 707 may be configured to: receiving a second sub-input of the user to the preview interface;

the display unit may also be used to:

in response to the second sub-input, a video recording interface is displayed.

In some embodiments, the processor 710 may also be configured to:

determining anti-shake intensity according to the N rotation matrixes;

In some embodiments, the processor 710 may also be configured to:

determining average similarity according to the N-1 rotation matrix similarity;

and determining the anti-shake strength according to the average similarity.

In some embodiments, the processor 710 may also be configured to:

It should be appreciated that in embodiments of the present application, the input unit 704 may include a graphics processor (Graphics Processing Unit, GPU) 7041 and a microphone 7042, with the graphics processor 7041 processing image data of still pictures or video obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The display unit 706 may include a display panel 7061, and the display panel 7061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 707 includes at least one of a touch panel 7071 and other input devices 7072. The touch panel 7071 is also referred to as a touch screen. The touch panel 7071 may include two parts, a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and so forth, which are not described in detail herein.

The memory 709 may be used to store software programs as well as various data. The memory 709 may mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area may store an operating system, application programs or instructions (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like. Further, the memory 709 may include volatile memory or nonvolatile memory, or the memory 709 may include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM), static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (ddr SDRAM), enhanced SDRAM (Enhanced SDRAM), synchronous DRAM (SLDRAM), and Direct RAM (DRRAM). Memory 709 in embodiments of the present application includes, but is not limited to, these and any other suitable types of memory.

Processor 710 may include one or more processing units; optionally, processor 710 integrates an application processor that primarily processes operations involving an operating system, user interface, application programs, and the like, and a modem processor that primarily processes wireless communication signals, such as a baseband processor. It will be appreciated that the modem processor described above may not be integrated into the processor 710.

The embodiment of the application further provides a readable storage medium, on which a program or an instruction is stored, where the program or the instruction realizes each process of the video anti-shake method embodiment when executed by a processor, and the same technical effects can be achieved, so that repetition is avoided, and no further description is given here.

Wherein the processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes computer readable storage medium such as computer readable memory ROM, random access memory RAM, magnetic or optical disk, etc.

The embodiment of the application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running a program or an instruction, implementing each process of the video anti-shake method embodiment, and achieving the same technical effect, so as to avoid repetition, and no redundant description is provided herein.

It should be understood that the chips referred to in the embodiments of the present application may also be referred to as system-on-chip chips, chip systems, or system-on-chip chips, etc.

The embodiments of the present application provide a computer program product stored in a storage medium, where the program product is executed by at least one processor to implement the respective processes of the embodiments of the video anti-shake method, and achieve the same technical effects, and are not repeated herein.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solutions of the present application may be embodied essentially or in a part contributing to the prior art in the form of a computer software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk), comprising several instructions for causing a terminal (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the methods described in the embodiments of the present application.

The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those of ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are also within the protection of the present application.

Claims

1. A video anti-shake method, comprising:

performing anti-shake processing on the at least two frames of images based on a second anti-shake parameter to obtain a target video;

2. The method of claim 1, wherein the anti-shake control comprises a control body and an identifier located on the control body; the second input includes a first sub-input to the identification;

the displaying a video recording interface in response to a second input from the user to the preview interface, comprising:

Determining a first anti-shake parameter and a second anti-shake parameter corresponding to the first sub-input in response to the first sub-input of the identifier by a user, wherein the first anti-shake parameter and the second anti-shake parameter have an association relationship with the input parameter of the first sub-input;

and displaying a video recording interface.

3. The method of claim 2, wherein the second input comprises a second sub-input, the displaying further comprising, prior to the video recording interface:

receiving a second sub-input of the user to the preview interface;

the display video recording interface comprises:

and displaying a video recording interface in response to the second sub-input.

4. The method of claim 2, wherein the determining the first anti-shake parameter and the second anti-shake parameter corresponding to the first sub-input comprises:

determining N rotation matrixes corresponding to the N frames of preview images according to the gyroscope sensor data corresponding to the N frames of preview images;

determining anti-shake intensity according to the N rotation matrixes;

5. The method of claim 4, wherein determining the anti-shake intensity from the N rotation matrices comprises:

based on the rotation matrixes corresponding to the two adjacent frames of preview images, calculating to obtain N-1 rotation matrix similarity;

determining average similarity according to the N-1 rotation matrix similarity;

and determining the anti-shake strength according to the average similarity.

6. The method of claim 5, wherein determining an average similarity from the N-1 rotation matrix similarities comprises:

and carrying out weighted average on the N-1 rotation matrix similarity to obtain the average similarity.

7. The method of claim 4, wherein determining the first anti-shake parameter and the second anti-shake parameter based on the anti-shake intensity comprises:

8. The method of claim 4, wherein the first sub-input is a sliding input, the first time period is determined based on input parameters of the first sub-input, the input parameters including at least one of: sliding distance, input time.

9. A video anti-shake apparatus, comprising:

the first display module is used for responding to a first input of a user and displaying a preview interface, wherein the preview interface comprises an anti-shake control, and the anti-shake control is used for adjusting anti-shake parameters;

the processing module is used for carrying out anti-shake processing on the at least two frames of images based on the second anti-shake parameters to obtain a target video;

10. The apparatus of claim 9, wherein the anti-shake control comprises a control body and an identifier located on the control body; the second input includes a first sub-input to the identification;

the second display module includes:

a determining unit, configured to determine a first anti-shake parameter and a second anti-shake parameter corresponding to a first sub-input in response to a first sub-input of the identifier by a user, where the first anti-shake parameter and the second anti-shake parameter have an association relationship with an input parameter of the first sub-input;

and the display unit is used for displaying the video recording interface.