WO2021184812A1 - 一种视频稳化方法及装置 - Google Patents

一种视频稳化方法及装置 Download PDF

Info

Publication number
WO2021184812A1
WO2021184812A1 PCT/CN2020/130572 CN2020130572W WO2021184812A1 WO 2021184812 A1 WO2021184812 A1 WO 2021184812A1 CN 2020130572 W CN2020130572 W CN 2020130572W WO 2021184812 A1 WO2021184812 A1 WO 2021184812A1
Authority
WO
WIPO (PCT)
Prior art keywords
stabilized
video
value
video frame
frame
Prior art date
Application number
PCT/CN2020/130572
Other languages
English (en)
French (fr)
Inventor
胡振邦
刘阳兴
Original Assignee
武汉Tcl集团工业研究院有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 武汉Tcl集团工业研究院有限公司 filed Critical 武汉Tcl集团工业研究院有限公司
Publication of WO2021184812A1 publication Critical patent/WO2021184812A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/21—Circuitry for suppressing or minimising disturbance, e.g. moiré or halo

Definitions

  • This application relates to the field of video processing technology, and in particular to a video stabilization method and device.
  • One of the objectives of the embodiments of the present application is to provide a video stabilization method and device, which aims to improve the problem of image distortion caused by shaking during video shooting.
  • the present application provides a video stabilization method, including: obtaining motion parameters of video frames other than the first frame in a video to be processed; and determining a stabilization value of a geometric parameter corresponding to the motion parameter according to the motion parameter ,
  • the geometric parameter is used to describe the degree of jitter of the non-first frame video frame; according to the transformation relationship between the motion parameter and the geometric parameter, the stabilized value of the geometric parameter is converted into a stabilized motion parameter Determine the stabilized video frame of the non-first video frame according to the stabilized motion parameter; generate the stabilized video corresponding to the video to be processed according to the stabilized video frame.
  • the video stabilization method provided in this application, by calculating the stabilization value of the geometric parameter corresponding to the motion parameter of each non-first frame video frame, the direct correction of the jitter degree of each non-first frame video frame is realized. To a certain extent, the picture distortion caused by the jitter of each frame other than the first video frame is eliminated. Therefore, it is ensured that the stabilized video obtained is compared with the video to be processed, and the picture distortion caused by jitter is eliminated to a certain extent.
  • the non-first video frame includes multiple frames, and the multiple non-first video frames are continuous; correspondingly, the generated video corresponding to the to-be-processed video frame is based on the stabilized video frame
  • the stabilized video includes: generating a stabilized video corresponding to the to-be-processed video according to the stabilized video frame corresponding to each of the non-first video frames of each frame.
  • the multiple non-first video frames are continuous with the first video frame of the to-be-processed video, and the stabilized video frames corresponding to each of the non-first video frames are generated according to the respective stabilized video frames.
  • the method further includes: determining the projective transformation matrix of the first video frame according to a preset cropping ratio and the height and width of the first video frame; The projective transformation matrix of the first video frame crops the first video frame to obtain the stabilized video frame of the first video frame; correspondingly, according to the multiple non-first video frames corresponding to each Stabilizing the video frame to generate the stabilizing video corresponding to the to-be-processed video includes: stabilizing the video frame according to the first video frame and the stabilizing video frame corresponding to each of the non-first video frames To generate a stabilized video corresponding to the to-be-processed video.
  • the determining a stabilization value of a geometric parameter corresponding to the motion parameter according to the motion parameter includes: converting the motion parameter into the geometric parameter according to the transformation relationship; The parameters are smoothed and filtered to obtain the stabilized values of the geometric parameters.
  • the geometric parameters include a first zoom ratio in a first direction, a second zoom ratio in a second direction, a first angle of deflection from the first direction, and a deviation from the second direction The second deflection angle, the first offset in the first direction, and the second offset in the second direction, wherein the first direction and the second direction are perpendicular to each other.
  • the performing smoothing filtering processing on the geometric parameter to obtain the stabilized value of the geometric parameter includes: performing smoothing filtering processing on the first zoom ratio and the second zoom ratio to obtain the The stabilization value of the first zoom ratio and the stabilization value of the second zoom ratio; smooth filtering processing is performed on the first deflection angle and the second deflection angle to obtain the stabilization of the first deflection angle Value and the stabilized value of the second deflection angle; perform smoothing filter processing on the first offset to obtain the stabilized value of the first offset; perform smooth filtering on the second offset Processing to obtain the stabilized value of the second offset.
  • the performing smoothing filtering processing on the first zoom ratio and the second zoom ratio to obtain the stabilized value of the first zoom ratio and the stabilized value of the second zoom ratio includes: The first zoom ratio and the second zoom ratio are brought into a preset first reversible function, and the mean value of the first zoom ratio and the second zoom ratio and the first preset mean value are calculated.
  • a degree of difference using a first smoothing filter to perform smoothing filtering processing on the first degree of difference to obtain a stabilized value of the first degree of difference; bringing the first zoom ratio and the second zoom ratio into In the preset second reversible function, a second degree of difference between the first zoom ratio and the second zoom ratio is calculated; a second smoothing filter is used to perform smoothing filtering processing on the second degree of difference, Obtain the stabilized value of the second degree of difference; using the stabilized value of the first degree of difference and the stabilized value of the second degree of difference to perform the first and second reversible functions Solve linearly to obtain the stabilized value of the first zoom ratio and the stabilized value of the second zoom ratio.
  • the smoothing filter processing is performed on the first deflection angle and the second deflection angle to obtain the stabilized value of the first deflection angle and the stabilized value of the second deflection angle, including : Bring the first deflection angle and the second deflection angle into a preset third reversible function, and calculate the average value of the first deflection angle and the second deflection angle and the third preset average value
  • the third degree of difference is smoothed and filtered by a third smoothing filter to obtain the stabilized value of the third degree of difference;
  • the first deflection angle and the second deflection angle are banded Into the preset fourth reversible function, the fourth degree of difference between the first deflection angle and the second deflection angle is calculated;
  • a fourth smoothing filter is used to smoothly filter the fourth degree of difference , Obtain the stabilized value of the fourth degree of difference; use the third degree of difference and the fourth degree of difference to linearly solve the third reversible function and the fourth reversible
  • the performing smoothing filtering processing on the first offset to obtain a stabilized value of the first offset includes: bringing the first offset into a preset fifth reversible Function to calculate the first ratio of the first offset to the frame width of the non-first video frame; use a fifth smoothing filter to perform smoothing filtering processing on the first ratio to obtain a first stabilization value
  • Use a sixth smoothing filter to perform smoothing filtering processing on the first ratio to obtain a second stabilized value perform weighted calculation on the first stabilized value and the second stabilized value to obtain the first The stabilization value of the offset; wherein the state transition noise matrix parameter of the fifth smoothing filter is smaller than the state noise matrix parameter of the sixth smoothing filter.
  • the performing a weighted calculation on the first stabilization value and the second stabilization value to obtain the stabilization value of the first offset includes: according to the value of the fifth smoothing filter
  • the state transition noise matrix calculates the weighted value corresponding to the fifth smoothing filter; calculates the weighted value corresponding to the sixth smoothing filter according to the weighted value corresponding to the fifth smoothing filter;
  • the weighted value corresponding to the fifth smoothing filter and the weighted value corresponding to the sixth smoothing filter are weighted and calculated on the first stabilized value and the second stabilized value to obtain the first offset
  • the stabilized value of the quantity includes: according to the value of the fifth smoothing filter
  • the state transition noise matrix calculates the weighted value corresponding to the fifth smoothing filter; calculates the weighted value corresponding to the sixth smoothing filter according to the weighted value corresponding to the fifth smoothing filter;
  • the weighted value corresponding to the fifth smoothing filter and the weighted value corresponding to the sixth smoothing filter are weight
  • the calculating the weighting value corresponding to the fifth smoothing filter according to the state transition noise matrix of the fifth smoothing filter includes: using a formula Calculating a weighting value corresponding to the fifth smoothing filter;
  • Q 5 represents the state transition noise matrix of the fifth smoothing filter
  • the function Express The absolute value of, dx norm represents the first ratio, Represents the first stabilization value
  • the function ⁇ () represents the calculation of the degree of difference, 0 ⁇ w 5 ⁇ 1.
  • the performing smoothing filtering processing on the second offset to obtain a stabilized value of the second offset includes: bringing the second offset into a sixth invertible function, and calculating The second offset occupies the second proportion of the picture height of the non-first frame video frame; the seventh smoothing filter is used to perform smoothing filtering processing on the second proportion to obtain the third stabilization value; The smoothing filter performs smoothing filtering processing on the second ratio to obtain a fourth stabilization value; performing a weighted calculation on the third stabilization value and the fourth stabilization value to obtain the value of the second offset Stabilization value; wherein the state transition noise matrix parameter of the seventh smoothing filter is smaller than the state noise matrix parameter of the eighth smoothing filter.
  • the weighted calculation of the third stabilization value and the fourth stabilization value to obtain the stabilization value of the second offset includes: using the seventh smoothing filter
  • the state transition noise matrix calculates the weight value corresponding to the seventh smoothing filter; calculates the weight value corresponding to the eighth smoothing filter according to the weight value corresponding to the seventh smoothing filter;
  • the weighted value corresponding to the seventh smoothing filter and the weighted value corresponding to the eighth smoothing filter are weighted and calculated on the third stabilized value and the fourth stabilized value to obtain the second offset The stabilized value of the quantity.
  • said using the state transition noise matrix of the seventh smoothing filter to calculate the weighting value corresponding to the seventh smoothing filter includes: using a formula Calculate the weighted value w 7 corresponding to the seventh smoothing filter; where Q 7 represents the state transition noise matrix of the seventh smoothing filter, and the function Express The absolute value of dy norm represents the second ratio, represents the third stabilization value, 0 ⁇ w 7 ⁇ 1.
  • the determining the stabilized video frame of the non-first frame video frame according to the stabilized motion parameter includes: calculating the projective transformation matrix of the non-first frame video frame according to the stabilized motion parameter; using The projective transformation matrix crops the non-first frame video frame to obtain a stabilized video frame of the non-first frame video frame.
  • the cutting the non-first frame video frame by using the projective transformation matrix to obtain the stabilized video frame of the non-first frame video frame includes: using the projective transformation matrix to perform the processing on the non-first frame
  • the frame video frame is cropped to obtain a stabilized sub-video frame; if the size of the stabilized sub-video frame is the same as the size of the non-first video frame, it is determined that the stabilized sub-video frame is the non- Stabilized video frame of the first video frame.
  • the method further includes: if the size of the stabilized sub-video frame is equal to the size of the stabilized sub-video frame. If the size of the non-first frame video frame is different, the projective transformation matrix is adjusted; the non-first frame video frame is cropped according to the adjusted projective transformation matrix to obtain the non-first frame video frame Stabilize the video frame.
  • the present application provides a video processing device, including: an acquisition unit for acquiring motion parameters, where the motion parameters are motion parameters of non-first frame and non-first frame video frames in the video to be processed; and a calculation unit for Calculate the stabilization value of the geometric parameter corresponding to the motion parameter according to the motion parameter, the geometric parameter is used to describe the jitter degree of the non-first frame video frame; and according to the difference between the motion parameter and the geometric parameter
  • the transformation relationship between the two is to convert the stabilized value of the geometric parameter into a stabilized motion parameter; the stabilizing unit is configured to determine the stabilized video frame of the non-first frame video frame according to the stabilized motion parameter.
  • the present application provides a video processing device, a processor, a memory, and a computer program stored in the memory and capable of running on the processor.
  • the processor executes the computer program, the following The method described in one aspect or any alternative of the first aspect.
  • the fourth unit provides a computer-readable storage medium
  • the computer-readable storage medium stores a computer program
  • the computer program when executed by a processor, it implements any optional manner such as the first aspect or the first aspect The method described.
  • the embodiments of the present application provide a computer program product, which when the computer program product runs on a video processing device, causes the video processing device to execute the steps of the video quality evaluation method described in the first aspect.
  • FIG. 1 is a schematic diagram of a video stabilization principle provided by an embodiment of the present application
  • FIG. 2 is a schematic flowchart of an embodiment of a video stabilization method provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of a transformation relationship between geometric parameters and motion parameters provided by an embodiment of the present application
  • FIG. 4 is a schematic diagram of comparison between the original video frame of test video 1 and the corresponding stabilized video frame provided by an embodiment of the present application;
  • FIG. 5 is a schematic diagram showing the comparison of the change curves of the parameter a and the stabilized value of the parameter a in the motion parameters of the test video 1 provided by the embodiment of the present application;
  • FIG. 6 is a schematic diagram of the comparison of the change curves of the parameter b and the stabilized value of the parameter b in the motion parameters of the test video 1 provided by the embodiment of the present application;
  • FIG. 7 is a schematic diagram showing the comparison of the change curves of the parameter c and the stabilized value of the parameter c in the motion parameters of the test video 1 provided by the embodiment of the present application;
  • FIG. 8 is a schematic diagram of the comparison of the change curves of the parameter d and the stabilized value of the parameter d in the motion parameters of the test video 1 provided by the embodiment of the present application;
  • FIG. 9 is a schematic diagram showing the comparison of the change curves of the parameter e and the stabilized value of the parameter e in the motion parameters of the test video 1 provided by the embodiment of the present application;
  • FIG. 10 is a schematic diagram of the comparison of the change curves of the parameter f and the stabilized value of the parameter f in the motion parameters of the test video 1 provided by the embodiment of the present application;
  • FIG. 11 is a schematic diagram of comparison between the original video frame of test video 2 and the corresponding stabilized video frame provided by an embodiment of the present application;
  • FIG. 12 is a schematic diagram of the comparison of the change curves of the parameter a and the stabilized value of the parameter a in the motion parameters of the test video 2 provided by an embodiment of the present application;
  • FIG. 13 is a schematic diagram of the comparison of the change curves of the parameter b and the stabilized value of the parameter b in the motion parameters of the test video 2 provided by the embodiment of the present application;
  • FIG. 14 is a schematic diagram of the comparison of the change curves of the parameter c and the stabilized value of the parameter c in the motion parameters of the test video 2 provided by the embodiment of the present application;
  • 15 is a schematic diagram showing the comparison of the change curves of the parameter d and the stabilized value of the parameter d in the motion parameters of the test video 2 provided by the embodiment of the present application;
  • 16 is a schematic diagram of the comparison of the change curves of the parameter e and the stabilized value of the parameter e in the motion parameters of the test video 2 provided by the embodiment of the present application;
  • FIG. 17 is a schematic diagram showing the comparison of the change curves of the parameter f and the stabilized value of the parameter f in the motion parameters of the test video 2 provided by the embodiment of the present application;
  • 18 is a schematic diagram of comparison between the original video frame of test video 3 and the corresponding stabilized video frame provided by an embodiment of the present application;
  • 19 is a schematic diagram of the comparison of the change curves of the parameter a and the stabilized value of the parameter a in the motion parameters of the test video 3 provided by the embodiment of the present application;
  • 20 is a schematic diagram of the comparison of the change curves of the parameter b and the stabilized value of the parameter b in the motion parameters of the test video 3 provided by the embodiment of the present application;
  • 21 is a schematic diagram of the comparison of the change curves of the parameter c and the stabilized value of the parameter c in the motion parameters of the test video 3 provided by the embodiment of the present application;
  • 22 is a schematic diagram showing the comparison of the change curves of the parameter d and the stabilized value of the parameter d in the motion parameters of the test video 3 provided by the embodiment of the present application;
  • FIG. 23 is a schematic diagram of the comparison of the change curves of the parameter e and the stabilized value of the parameter e in the motion parameters of the test video 3 provided by the embodiment of the present application;
  • 24 is a schematic diagram of the comparison of the change curves of the parameter f and the stabilized value of the parameter f in the motion parameters of the test video 3 provided by an embodiment of the present application;
  • FIG. 25 is a schematic structural diagram of a video stabilization device provided by an embodiment of the present application.
  • FIG. 26 is a schematic structural diagram of a video processing device provided by an embodiment of the present application.
  • FIG. 27 is a schematic diagram of a system architecture of a video processing system provided by an embodiment of the present application.
  • the video frame in the to-be-processed video shot by the video processing device is denoted by F.
  • the first video frame (ie, the first video frame) of the video to be processed is denoted as F 1
  • the second video frame is denoted as F 2
  • the third video frame is denoted as F 3
  • the fourth The video frame is represented as F 4 , and so on.
  • the projective transformation relationship can be determined by the motion parameter To represent. That is, the motion parameter M is used to represent the projective transformation relationship between two adjacent video frames.
  • M can be calculated based on motion estimation.
  • Conventional motion estimation methods include two kinds. One is that the video processing device directly performs motion estimation based on the image content of F t and F t-1 to obtain M t . The other is that the video processing device obtains data from an auxiliary sensor (for example, a gyroscope) to perform motion estimation on the image content of F t and F t-1 to obtain M t .
  • an auxiliary sensor for example, a gyroscope
  • the obtained stabilized video frame of each video frame is denoted by S.
  • the stabilized video frame of F 1 is denoted as S 1
  • the stabilized video frame of F 2 is denoted as S 2
  • the stabilized video frame of F 3 is denoted as S 3
  • the stabilized video frame of F 4 is denoted Is S 4 , and so on.
  • the projective transformation relationship between two adjacent stabilized video frames can also be represented by the stabilized motion parameter K.
  • Image content and stability of the t-1 frame video image S t-1 of the contents of each other can be based on stabilization of the motion parameter K t dimensional plane projective transformation approximated Means that S t ⁇ f(S t-1 ,K t ),
  • K t is an ideal constant velocity motion parameter obtained after stabilizing M t.
  • the subject of the video stabilization method provided in this application is a video processing device.
  • the video processing device may be a terminal device (including but not limited to mobile terminals such as smart phones, tablet computers, wearable devices, etc., but also cameras, robots, etc. in various application scenarios), or a server. That is, the video stabilization method provided in this application can be executed by a terminal device or a server.
  • a video processing system provided for this application includes at least one terminal device and a server. After the terminal device obtains the video to be processed, it can send the video to be processed to the server.
  • the server stabilizes the to-be-processed video by executing the video stabilization method provided in this application to obtain a stabilized video of the to-be-processed video. The server then sends the stabilized video to the terminal device.
  • the terminal device may stabilize the to-be-processed video by executing the video stabilization method provided in this application to obtain a stabilized video of the to-be-processed video.
  • FIG. 2 is a schematic flowchart of a video stabilization method provided by an embodiment of the present application.
  • the video stabilization method shown in Figure 2 may include:
  • S201 Acquire motion parameters of video frames other than the first frame in the to-be-processed video.
  • any video frame other than the first frame (ie, the first frame) video frame in the video to be processed is a non-first frame video frame.
  • the video to be processed includes a total of 10 video frames, and the second frame to the tenth frame are all non-first video frames.
  • the video processing device may perform motion estimation after acquiring the non-first video frame and the previous video frame whose playback time is earlier than the non-first video frame to obtain the motion parameters of the non-first video frame.
  • the video processing apparatus may F T and F t-1 motion estimation image content, motion parameter obtained F T M t.
  • S202 Determine, according to the motion parameter, a stabilized value of the geometric parameter corresponding to the motion parameter.
  • the degree of jitter of the video frame can be described by defining geometric parameters, that is, the geometric parameter can be understood as a parameter used to describe the degree of jitter of the video frame.
  • the geometric parameters may include the degree of zoom, the degree of deflection, and/or the degree of offset of the video frame.
  • the degree of jitter of a video frame can be understood as the degree of jitter between the video frame and adjacent video frames or other video frames, that is, the relative degree of jitter between the video frame and adjacent video frames or other video frames. In terms of the overall image offset, zoom, and deflection.
  • the non-first video frame has jittered, and if the non-first video frame is relative to the When the overall displacement between adjacent video frames is less than or equal to the preset offset threshold, it can be considered that the non-first frame video frame has achieved stabilization.
  • the zooming degree the larger the zooming degree of the overall picture that occurs between the non-first frame video frame and its adjacent video frames, the greater the jitter degree of the non-first frame video frame.
  • the non-first frame video frame and its relative The smaller the zooming degree of the entire picture that occurs between adjacent video frames, the smaller the jitter degree of the non-first video frame.
  • the zooming degree of the entire picture between the non-first video frame and its adjacent video frames is greater than the preset zoom threshold, it can be considered that the non-first video frame has jittered, and if the non-first video frame is relative to the When the overall zooming degree between adjacent video frames is less than or equal to the preset partial zoom value, it can be considered that the non-first video frame has achieved stabilization.
  • the degree of deflection the greater the overall picture deflection that occurs between the non-first frame of video frame and its adjacent video frames, the greater the degree of jitter of the non-first frame of video frame, on the contrary, the non-first frame of video frame and its adjacent.
  • the overall picture deflection that occurs between video frames the smaller the jitter degree of the non-first frame video frame. It is understandable that if the overall deflection of the picture between the non-first video frame and its neighboring video frames is greater than the preset deflection angle threshold, it can be considered that the non-first video frame has jittered. When the overall deflection that occurs between adjacent video frames is less than or equal to the preset deflection angle threshold, it can be considered that the non-first frame video frame has achieved stabilization.
  • the mutual conversion between the geometric parameters and the motion parameters is realized by establishing the transformation relationship between the geometric parameters and the motion parameters.
  • the video to be processed is composed of two-dimensional video frames frame by frame. Therefore, the geometric parameters of each video frame other than the first frame can be defined in two mutually perpendicular directions (assumed to be the first direction and the second direction).
  • the geometric parameters may include a first zoom ratio V x in the first direction, a second zoom ratio V y in the second direction, a first deflection angle ⁇ x deviating from the first direction, and a first deflection angle ⁇ x deviating from the first direction.
  • V x and V y can describe the degree of zoom of the video frame
  • ⁇ x and ⁇ y can describe the degree of deflection of the video frame
  • dx and dy can describe the degree of deviation of the video frame.
  • the coordinate system formed O xy perpendicular to each other to establish the origin of the first coordinate axis direction (x-axis) and the axis (y-axis) in a second direction, x-axis and y-axis as point O, the coordinate system formed O xy.
  • Oxy is used as the preset reference coordinates, and the video processing device uses the degree of difference between the actual coordinate system of the video frame and the coordinate system Oxy to measure the degree of jitter of the video frame.
  • the x-axis rotates according to the first deflection angle ⁇ x , and expands and contracts based on the first zoom ratio V x to form an x1 axis.
  • the y axis rotates according to the second deflection angle ⁇ y , and expands and contracts based on the second zoom ratio V y to form a y1 axis.
  • the x1 axis is offset along the y1 axis by the second offset amount dy to form an x2 axis
  • the y1 axis is offset along the x1 axis direction by the first offset amount dx to form the y2 axis.
  • the intersection point B of the x2 axis and the y2 axis is the coordinate origin of the x2 axis and the y2 axis.
  • the coordinate system B x2-y2 formed by the origin B, the x2-axis and the y2-axis is the actual coordinate system of the video frame.
  • Affine transformation is performed on the reference coordinate system Oxy and the coordinate system B x2-y2.
  • the six-element equation system is used as the transformation relationship between M and geometric parameters to realize the mutual transformation between M and geometric parameters. That is, through the geometric transformation relationship, the geometric parameters that can reflect the jitter degree of the non-first video frame can be directly obtained based on the motion parameters.
  • the video processing device may convert the motion parameters of the non-first video frame into corresponding geometric parameters based on the above transformation relationship (ie, the transformation relationship between M and geometric parameters).
  • each parameter value (ie, a, b, c, d, e, f) in the motion parameter M of different non-first frame video frames may be different. Therefore, the different M is substituted into the above six-element equations Later, the geometric parameters obtained are also different.
  • the video processing device After the video processing device obtains the geometric parameter corresponding to the motion parameter of the video frame other than the first frame, it can perform smoothing filtering processing on the geometric parameter to obtain the stabilized value of the geometric parameter.
  • the video processing device can smooth and filter the geometric parameters based on the Kalman filter to eliminate the noise of the geometric parameters and reduce sudden changes, so that the stabilized value of the geometric parameters is in the previous video frame adjacent to the non-first video frame. Stable changes on the basis.
  • geometric parameters are parameters that can reflect the degree of jitter of non-first frame video frames, by smoothing and filtering the geometric parameters, the jitter of geometric parameters can be eliminated, and the stabilization of geometric parameters can be achieved. It can further eliminate picture distortion caused by jitter and ensure the stabilization effect of non-first frame video frames.
  • the so-called stabilization of the non-first frame video frame refers to making the non-first frame video frame present a relatively stable picture, and to a certain extent eliminates picture distortion caused by jitter.
  • the video processing device may directly use the smoothing filter to perform smoothing filtering processing on each geometric parameter to obtain the stabilized value of the geometric parameter.
  • the smoothing filter may be a Kalman filter.
  • the video processing device may divide the geometric parameters corresponding to the non-first video frame into four groups for filtering.
  • the four sets of filtering are:
  • V x and V y of the smoothing filter process to obtain stable values V y of stabilization of the value of V x V x and V y in K K.
  • the video processing device can be implemented through the following steps S01-S05:
  • V c can take the value 1, and can also take the value of the average value of V x and V y of the previous video frame F t-1 adjacent to the non-first video frame.
  • first smoothing filter and the second smoothing filter may be Kalman filters with the same parameters.
  • the combined filtering of the values of ⁇ x and ⁇ y can effectively improve the distortion caused by the non-first frame video frame when the coordinate axis is rotated in an unequal ratio.
  • the video processing device can be implemented through the following steps S06-S10:
  • ⁇ c can take a value of 0, or can take the value of the average value of ⁇ x and ⁇ y of the previous video frame adjacent to the non-first video frame.
  • the third smoothing filter and the fourth smoothing filter may be Kalman filters with the same parameters.
  • V x , V y , ⁇ x , and ⁇ y characterize the rotation and scaling of the picture of the video frame, it has nothing to do with the size of the video frame, so the stabilization is directly based on the smoothing filter.
  • dx and dy are displacements, and the unit is pixel, which is related to the size of the video frame. Therefore, before stabilizing dx and dy, a normalization operation needs to be performed first.
  • the video processing device can be implemented through the following steps S11-S14:
  • dx norm is the normalized value of dx.
  • both the fifth smoothing filter and the sixth smoothing filter may be Kalman filters. And the conversion of the fifth smoothing filter and the sixth smoothing filter is exactly the same as the observed noise matrix parameter, but the state transition noise matrix parameter of the fifth smoothing filter is smaller than the state noise matrix parameter of the sixth smoothing filter.
  • the video processing device may control whether dx is eliminated based on the weights of the fifth smoothing filter and the sixth smoothing filter.
  • the fifth smoothing filter can correspond to the horizontal jitter that needs to be eliminated.
  • the sixth smoothing filter can correspond to the user's deliberate movement. For example, when the movement amplitude is large, that is, when the dx norm is large, it is likely to be caused by deliberate movement of the user. At this time, it can be considered that dx is a horizontal jitter that does not need to be eliminated.
  • the video processing device may calculate the weight value w 5 corresponding to the fifth smoothing filter according to the state transition noise matrix Q 5 of the fifth smoothing filter; according to w 5 , calculate and The weighted value w 6 corresponding to the sixth smoothing filter; using w 5 and w 6 to with Perform weighting calculation to get dx K.
  • the video processing device may specifically calculate w 5 using the following formula:
  • the function Express The absolute value of, the function ⁇ () represents the calculation of the degree of difference, 0 ⁇ w 5 ⁇ 1.
  • dx K is the weighted stabilization value of dx after normalization.
  • the video processing device can be implemented through the following steps S15-S18:
  • both the seventh smoothing filter and the eighth smoothing filter may be Kalman filters. And the conversion of the fifth smoothing filter and the eighth smoothing filter is exactly the same as the observed noise matrix parameter, but the state transition noise matrix parameter of the seventh smoothing filter is smaller than the state noise matrix parameter of the eighth smoothing filter.
  • the video processing device may control whether dy is eliminated based on the weights of the seventh smoothing filter and the eighth smoothing filter.
  • the seventh smoothing filter can correspond to the vertical jitter that needs to be eliminated.
  • the eighth smoothing filter can correspond to the user's deliberate movement. For example, when the motion amplitude is large, that is, when the dy norm is large, it is likely to be caused by the user's deliberate movement. At this time, it can be considered that dy does not need to be eliminated.
  • Vertical jitter when the motion amplitude is small, that is, when the dy norm is small, it can be considered that dy is the vertical jitter that needs to be eliminated.
  • the eighth smoothing filter can correspond to the user's deliberate movement. For example, when the motion amplitude is large, that is, when the dy norm is large, it is likely to be caused by the user's deliberate movement. At this time, it can be considered that dy does not need to be eliminated. Vertical jitter.
  • the video processing device may use the state transition noise matrix Q 7 of the seventh smoothing filter to calculate the weighting value w 7 corresponding to the seventh smoothing filter; and then calculate according to w 7 The weighted value w 8 corresponding to the eighth smoothing filter; finally, w 7 and w 8 are used to with Perform a weighted calculation to obtain dy K.
  • the video processing device can specifically use public Calculate w 7 .
  • the function Express The absolute value of 0 ⁇ w 7 ⁇ 1.
  • dy K is the weighted stable value of dy after normalization.
  • S203 Calculate the stabilized motion parameter of the motion parameter according to the transformation relationship between the motion parameter and the corresponding geometric parameter and the stabilized value of the geometric parameter.
  • the stabilized value of each geometric parameter is re-introduced into the above-mentioned six-element equation system for linear solution to the stabilized motion parameter of the motion parameter.
  • the stabilization value of the geometric parameter corresponding to M t is brought into the above-mentioned six-element equation set for linear solution, the stabilization value of a in M t a K , the stabilization value of b b K , and the stabilization value of c are obtained.
  • S204 Determine a stabilized video frame that is not the first video frame according to the stabilized motion parameter.
  • the video processing device may calculate the projective transformation matrix of the non-first frame video frame according to the stabilization motion parameter.
  • the non-first video frame is F t and the motion parameter is M t
  • the stabilized motion parameter of the motion parameter is K t .
  • video processing equipment can use the formula The projective transformation matrix R t of F t is calculated.
  • the first frame of the video to be processed does not have motion parameters, so the video processing device can cut the center of the matrix As R F 1 1.
  • r represents a predetermined proportion of cropping
  • W represents a width of F
  • H represents high F 1.
  • the non-first frame video frame can be cropped using the projective transformation matrix to obtain a stabilized video frame of the non-first frame video frame.
  • the projective transformation matrix may be used to obtain the image of some pixels from the non-first video frame, that is, the image is out of bounds.
  • the video processing device uses the projective transformation matrix to trim the non-first frame video frame to obtain the stabilized video frame, it may first use the projective transformation matrix to crop the non-first frame video frame to obtain the stabilized video frame. Sub video frame. If the size of the stabilized sub-video frame is the same as the size of the non-first video frame, it is determined that the stabilized sub-video frame is a stabilized video frame of the non-first video frame. The size of the stabilized sub-video frame is the same as the size of the non-first video frame, indicating that the projective transformation matrix does not cause the problem of image out-of-bounds in the stabilized video frame.
  • the video processing device can adjust the projective transformation matrix. Then, the non-first frame video frame is cropped according to the adjusted projective transformation matrix to obtain a stabilized video frame of the non-first frame video frame.
  • the length in the first direction in the video frame represents the width of the video frame, and the two sides perpendicular to the first direction are the left side and the right side, respectively.
  • the length of the second direction in the video frame represents the height of the video frame, and the two sides perpendicular to the second direction are the upper side and the lower side, respectively.
  • F t takes F t as an example of a video frame other than the first frame.
  • the terminal When the width of the stabilized sub-video frame is less than the width of F t , the terminal first determines which side is out of bounds. For example, if the pixels on the right side of the video frame stability facilitator R t indicated there is no corresponding pixel in F t, i.e. F t can not be extracted in the image corresponding to the content, the video frame indicates steady facilitator The right side crosses the line.
  • the video processing device can adjust R t so that the right side of the stabilized sub-video frame cropped according to the adjusted R t is aligned with the right side of F t. That is, the pixels on the right side of the stabilized sub-video frame obtained by cropping according to the adjusted R t correspond to the pixels on the left side of F t.
  • the said stable video facilitator can adjust R t so that the left side of the stabilized sub-video frame cropped according to the adjusted R t is aligned with the left side of F t. That is, the pixels on the right side of the stabilized sub-video frame obtained by cropping according to the adjusted R t correspond to the pixels on the left side of F t.
  • the video processing device can adjust R t by adjusting the scaling ratios in the first direction and the second direction in the stabilized value of the geometric parameter, so that according to the adjustment
  • the width of the stabilized sub-video frame obtained by the subsequent R t cropping is the same as the width of F t.
  • the video processing device can adjust R t so that the upper edge of the stabilized sub-video frame cropped according to the adjusted R t is aligned with the upper edge of F t. That is, the pixel points on the upper edge of the stabilized sub-video frame obtained by cropping according to the adjusted R t correspond to the pixels on the upper edge of F t.
  • the said stable video facilitator can adjust R t so that the bottom edge of the stabilized sub-video frame cropped according to the adjusted R t is aligned with the bottom edge of F t. That is, the pixels on the lower edge of the stabilized sub-video frame obtained by cropping according to the adjusted R t correspond to the pixels on the edge of F t.
  • the video processing device can adjust R t by adjusting the zoom ratios in the first direction and the second direction in the stabilized value of the geometric parameter, so that according to the adjustment
  • the height of the stabilized sub-video frame obtained by the subsequent R t cropping is the same as the height of F t.
  • the stabilization motion parameter of the motion parameter is obtained, which can realize the direct correction of the jitter degree of the non-first frame video frame, and ensure that the The stabilization effect of the first video frame can effectively improve the distortion of the picture caused by the jitter of the non-first video frame.
  • the video processing device can process a non-first frame video frame in the video to be processed according to actual needs, or can process multiple non-first frame video frames in the video frame to be processed.
  • the number of non-first video frames is not limited in.
  • S205 Generate a stabilized video corresponding to the to-be-processed video according to the stabilized video frame of the non-first video frame.
  • the stabilized video is composed of stabilized video frames obtained by the video processing device after stabilizing at least one video frame in the video to be processed.
  • the at least one video frame may be a continuous non-first video frame, or may be a continuous video frame including the first video frame.
  • the video processing device can perform stabilization processing on the entire video to be processed, or it can perform stabilization processing on a video segment composed of at least one continuous video frame in the video to be processed.
  • the duration of the video to be processed is 1 minute
  • the playback time can be all the videos between the 10s and 20s.
  • the video clip formed by the frame is used as the video clip that needs to be video stabilized.
  • the user can directly perform stabilization processing on the to-be-processed video.
  • the video processing device can perform each frame of non-first frame video frame based on the above steps S201-S204. Stabilize, get the stabilized video frame of each frame other than the first video frame.
  • the video processing device may generate a stabilized video corresponding to the to-be-processed video according to the stabilized video frame corresponding to each of the non-first video frames of each frame.
  • the video processing device can base on the above steps S201-S204 to be processed or each frame of the video segment that is not the first frame
  • the video frame is stabilized, and a stabilized video frame of each frame other than the first video frame is obtained.
  • the video processing device may generate the stabilized video corresponding to the to-be-processed video according to the stabilized video frame corresponding to the first video frame and each of the non-first video frames. video.
  • the jitter of non-first video frames captured subsequently can be understood as taking the image of the first video frame as a reference.
  • the jitter of the whole picture that occurs in the picture of the first video frame. Therefore, the video processing device can directly use the first video frame as the first video frame of the stabilized video corresponding to the video to be processed. And the stabilized video frame corresponding to the first video frame and each non-first video frame is determined as the stabilized video.
  • the video processing device may also stabilize the first video frame.
  • the video processing device uses the preset cropping ratio and the height and width of the first video frame to determine the projective transformation matrix of the first video frame (that is, the center cropping matrix calculated above). ), the first video frame may also be cropped according to the projective transformation matrix of the first video frame to obtain a stabilized video frame of the first video frame.
  • the video processing device can generate a stabilized video according to the stabilized video frame of the first video frame and the stabilized video frame of each non-first video frame.
  • the video stabilization method provided by this application can directly correct the jitter degree of each non-first frame video frame, to a certain extent, eliminate the jitter caused by each non-first frame video frame.
  • the resulting picture distortion Therefore, it is ensured that the stabilized video obtained is compared with the video to be processed, and the picture distortion caused by jitter is eliminated to a certain extent.
  • test video 1 has rotational motion during shooting.
  • FIG. 4 it is a schematic diagram of comparison between an original video frame in the test video 1 and a stabilized video frame corresponding to the original video frame.
  • FIG. 4 shows the movement trajectories of several points in the original video frame
  • FIG. 4 shows the movement trajectories of several points in the stabilized video frame. From the comparison of FIG. 4, it can be seen that after the test video is stabilized by the video stabilization method provided in this application, the jitter amplitude of each point is significantly reduced.
  • the change curve of parameter a is shown in Fig. 5(a), and the change curve of the stabilized value corresponding to parameter a is shown in Fig. 5(b).
  • the change curve of parameter b is shown in Fig. 6(a), and the change curve of the stabilized value corresponding to parameter b is shown in Fig. 6(b).
  • the change curve of parameter c is shown in Fig. 7(a), and the change curve of the stabilized value corresponding to parameter c is shown in Fig. 7(b).
  • the change curve of parameter d is shown in Fig.
  • test video 2 has vertical shaking motion during shooting.
  • FIG. 11 it is a schematic diagram of comparison between an original video frame in the test video 2 and a stabilized video frame corresponding to the original video frame.
  • FIG. 11 shows the movement trajectories of several points in the original video frame
  • FIG. 11 shows the movement trajectories of several points in the stabilized video frame. From the comparison of FIG. 11, it can be seen that after the test video 2 is stabilized by the video stabilization method provided by the present application, the vertical movement amplitude of each point is significantly reduced.
  • the change curve of parameter a is shown in Fig. 12(a), and the change curve of the stabilized value corresponding to parameter a is shown in Fig. 12(b).
  • the change curve of parameter b is shown in Fig. 13(a), and the change curve of the stabilized value corresponding to parameter b is shown in Fig. 13(b).
  • the change curve of parameter c is shown in Fig. 14(a), and the change curve of the stabilized value corresponding to parameter c is shown in Fig. 14(b).
  • the change curve of parameter d is shown in Fig.
  • the test video 3 has random jitters during the shooting process.
  • FIG. 18 it is a schematic diagram of comparison between an original video frame in the test video 3 and a stabilized video frame corresponding to the original video frame.
  • FIG. 18 shows the movement trajectories of several points in the original video frame
  • FIG. 18 shows the movement trajectories of several points in the stabilized video frame. From the comparison of FIG. 18, it can be seen that after the test video 3 is stabilized by the video stabilization method provided in this application, the motion amplitude of each point is significantly reduced.
  • the change curve of parameter a is shown in Fig. 19(a), and the change curve of the stabilized value corresponding to parameter a is shown in Fig. 19(b).
  • the change curve of parameter b is shown in Fig. 20(a), and the change curve of the stabilized value corresponding to parameter b is shown in Fig. 20(b).
  • the change curve of parameter c is shown in Fig. 21(a), and the change curve of the stabilized value corresponding to parameter c is shown in Fig. 21(b).
  • the change curve of parameter d is shown in Fig.
  • the embodiment of the present invention further provides an embodiment of an apparatus for implementing the foregoing method embodiment.
  • FIG. 25 is a schematic diagram of a video processing apparatus provided by an embodiment of the present application.
  • the included units are used to execute the steps in the embodiment corresponding to FIG. 2.
  • the video processing device 25 includes:
  • the acquiring unit 2501 is configured to acquire the motion parameters of the non-first frame video frame for each non-first frame video frame in the to-be-processed video.
  • the calculation unit 2502 is configured to calculate a stabilization value of a geometric parameter corresponding to the motion parameter according to the motion parameter, where the geometric parameter is used to describe the jitter degree of the non-first frame video frame; and according to the motion parameter
  • the transformation relationship between the geometric parameter and the geometric parameter converts the stabilized value of the geometric parameter into a stabilized motion parameter.
  • the stabilization unit 2503 is configured to determine the stabilized video frame of the non-first frame video frame according to the stabilization motion parameter, and according to the first frame video frame and each frame of the non-first frame video frame in the to-be-processed video Respectively corresponding to the stabilized video frames, and generate the stabilized video corresponding to the to-be-processed video.
  • the stabilization unit 2503 is further configured to generate the to-be-processed video frame according to the respective stabilized video frames corresponding to the first video frame and each non-first-frame video frame in the to-be-processed video.
  • the stabilization unit 2503 determines the projective transformation matrix of the first video frame according to the preset cropping ratio and the height and width of the first video frame of the to-be-processed video; and according to the first video frame
  • the first video frame is cropped by the projective transformation matrix to obtain a stabilized video frame of the first video frame.
  • the stabilization unit 2503 generates a stabilized video corresponding to the to-be-processed video according to the respective stabilization video frame corresponding to the first video frame and each non-first-frame video frame in the to-be-processed video, including : Generate a stabilized video corresponding to the to-be-processed video according to the stabilized video frame corresponding to each video frame in the to-be-processed video.
  • the calculation unit 2502 calculates the stabilization value of the geometric parameter corresponding to the motion parameter according to the motion parameter, including: converting the motion parameter into the geometric parameter according to the transformation relationship; The geometric parameter is smoothed and filtered to obtain the stabilized value of the geometric parameter.
  • the geometric parameters include a first zoom ratio in a first direction, a second zoom ratio in a second direction, a first angle of deflection from the first direction, and a deviation from the second direction The second deflection angle, the first offset in the first direction, and the second offset in the second direction, wherein the first direction and the second direction are perpendicular to each other.
  • the calculation unit 2502 performs smoothing filtering processing on the geometric parameter to obtain the stabilized value of the geometric parameter, including: performing smoothing filtering processing on the first zoom ratio and the second zoom ratio, Obtain the stabilized value of the first zoom ratio and the stabilized value of the second zoom ratio; perform smoothing filtering processing on the first deflection angle and the second deflection angle to obtain the first deflection angle The stabilization value of the second deflection angle and the stabilization value of the second deflection angle; the smoothing filtering process is performed on the first offset to obtain the stabilization value of the first offset; the second offset is Perform smoothing filtering processing to obtain the stabilized value of the second offset.
  • the calculation unit 2502 performs smoothing filtering processing on the first zoom ratio and the second zoom ratio to obtain a stabilized value of the first zoom ratio and a stabilized value of the second zoom ratio , Including: bringing the first zoom ratio and the second zoom ratio into a preset first reversible function, and calculating the mean value of the first zoom ratio and the second zoom ratio and the first preset The first difference degree of the mean value; the first degree of difference is smoothed and filtered by a first smoothing filter to obtain the stabilized value of the first degree of difference; the first scaling ratio and the second scaling The ratio is brought into the preset second reversible function, and the second degree of difference between the first zoom ratio and the second zoom ratio is calculated; the second degree of difference is smoothed by a second smoothing filter Filtering processing to obtain the stabilized value of the second degree of difference; using the stabilized value of the first degree of difference and the stabilized value of the second degree of difference to compare the first and second degree of difference
  • the reversible function is linearly solved to
  • the calculation unit 2502 performs smoothing filtering processing on the first deflection angle and the second deflection angle to obtain the stabilization value of the first deflection angle and the stabilization value of the second deflection angle
  • the value includes: bringing the first deflection angle and the second deflection angle into a preset third reversible function, and calculating the average value of the first deflection angle and the second deflection angle and the third preset Set the third difference degree of the mean value; use a third smoothing filter to perform smoothing filtering processing on the third difference degree to obtain the stabilized value of the third difference degree; compare the first deflection angle and the second The deflection angle is brought into the preset fourth reversible function, and the fourth degree of difference between the first deflection angle and the second deflection angle is calculated; a fourth smoothing filter is used to perform the fourth degree of difference Smoothing filter processing to obtain the stabilized value of the fourth degree of difference; using the third degree of difference and the fourth degree of difference to linearly solve the third revers
  • the calculation unit 2502 performs smoothing filtering processing on the first offset to obtain a stabilized value of the first offset, including: bringing the first offset into a preset
  • the fifth reversible function is to calculate the first ratio of the first offset to the frame width of the non-first frame of video; use a fifth smoothing filter to perform smoothing filtering processing on the first ratio to obtain a first Stabilized value; use a sixth smoothing filter to perform smoothing filtering processing on the first ratio to obtain a second stabilized value; perform weighted calculation on the first stabilized value and the second stabilized value to obtain the The stabilization value of the first offset; wherein the state transition noise matrix parameter of the fifth smoothing filter is smaller than the state noise matrix parameter of the sixth smoothing filter.
  • the calculation unit 2502 performs a weighted calculation on the first stabilization value and the second stabilization value to obtain the stabilization value of the first offset, including: according to the fifth smoothing
  • the state transition noise matrix of the filter calculates the weighted value corresponding to the fifth smoothing filter; calculates the weighted value corresponding to the sixth smoothing filter according to the weighted value corresponding to the fifth smoothing filter; using The weighted value corresponding to the fifth smoothing filter and the weighted value corresponding to the sixth smoothing filter are weighted and calculated on the first stabilized value and the second stabilized value to obtain the first stabilized value.
  • a stabilized value of the offset is a weighted calculation on the first stabilization value and the second stabilization value to obtain the stabilization value of the first offset, including: according to the fifth smoothing
  • the state transition noise matrix of the filter calculates the weighted value corresponding to the fifth smoothing filter; calculates the weighted value corresponding to the sixth smoothing filter according to the weighted value corresponding to the
  • the calculation unit 2502 calculates the weighting value corresponding to the fifth smoothing filter according to the state transition noise matrix of the fifth smoothing filter, including: using a formula Calculate the weighted value corresponding to the fifth smoothing filter; where Q 5 represents the state transition noise matrix of the fifth smoothing filter, and the function Express
  • Q 5 represents the state transition noise matrix of the fifth smoothing filter
  • the function ⁇ () represents the calculation of the degree of difference, 0 ⁇ w 5 ⁇ 1.
  • the calculation unit 2502 performs smoothing filtering processing on the second offset to obtain a stabilized value of the second offset, including: bringing the second offset into a sixth reversible Function to calculate the second ratio of the second offset occupying the height of the video frame of the non-first frame; use a seventh smoothing filter to perform smoothing filtering processing on the second ratio to obtain a third stabilization value; Use an eighth smoothing filter to perform smoothing filtering processing on the second ratio to obtain a fourth stabilized value; perform a weighted calculation on the third stabilized value and the fourth stabilized value to obtain the second bias The stabilization value of the shift amount; wherein the state transition noise matrix parameter of the seventh smoothing filter is smaller than the state noise matrix parameter of the eighth smoothing filter.
  • the calculation unit 2502 performs a weighted calculation on the third stabilization value and the fourth stabilization value to obtain the stabilization value of the second offset, including: using the seventh smoothing
  • the state transition noise matrix of the filter calculates the weighted value corresponding to the seventh smoothing filter; calculates the weighted value corresponding to the eighth smoothing filter according to the weighted value corresponding to the seventh smoothing filter; using The weighted value corresponding to the seventh smoothing filter and the weighted value corresponding to the eighth smoothing filter are weighted and calculated on the third stabilized value and the fourth stabilized value to obtain the first 2.
  • the stabilized value of the offset including: using the seventh smoothing
  • the state transition noise matrix of the filter calculates the weighted value corresponding to the seventh smoothing filter; calculates the weighted value corresponding to the eighth smoothing filter according to the weighted value corresponding to the seventh smoothing filter; using The weighted value corresponding to the seventh smoothing filter and the weighted value corresponding to the eighth smoothing filter are weight
  • the calculation unit 2502 using the state transition noise matrix of the seventh smoothing filter to calculate the weighting value corresponding to the seventh smoothing filter includes: using a formula Calculate the weighted value w 7 corresponding to the seventh smoothing filter; where Q 7 represents the state transition noise matrix of the seventh smoothing filter, and the function Express The absolute value of, dy norm represents the second ratio, Indicates the third stabilization value, 0 ⁇ w 7 ⁇ 1.
  • the stabilization unit 2503 determines the stabilized video frame of the non-first frame video frame according to the stabilized motion parameter, including: calculating the projection of the non-first frame video frame according to the stabilized motion parameter Transformation matrix; using the projective transformation matrix to crop the non-first frame video frame to obtain a stabilized video frame of the non-first frame video frame.
  • the stabilizing unit 2503 uses the projective transformation matrix to crop the non-first frame video frame to obtain the stabilized video frame of the non-first frame video frame, including: using the projective transformation matrix to The non-first video frame is cropped to obtain a stabilized sub-video frame; if the size of the stabilized sub-video frame is the same as the size of the non-first-frame video frame, the stabilized sub-video frame is determined Is a stabilized video frame other than the first video frame.
  • the stabilization unit 2503 uses the projective transformation matrix to crop the non-first video frame to obtain a stabilized sub-video frame, the method further includes: if the stabilized sub-video frame is If the size is different from the size of the non-first video frame, the projective transformation matrix is adjusted; the non-first video frame is cropped according to the adjusted projective transformation matrix to obtain the non-first video frame Stabilized video frame of frame video frame.
  • Fig. 26 is a schematic diagram of a video processing device provided by an embodiment of the present application.
  • the video processing device 26 of this embodiment includes a processor 260, a memory 261, and a computer program 262 stored in the memory 261 and running on the processor 260, such as a voice recognition program.
  • the processor 260 implements the steps in the foregoing embodiments of the video stabilization method when the computer program 262 is executed, for example, steps 201-204 shown in FIG. 2.
  • the processor 260 executes the computer program 262
  • the functions of the modules/units in the foregoing device embodiments such as the functions of the units 2501-2503 shown in FIG. 24, are realized.
  • the computer program 262 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 261 and executed by the processor 260 to complete the application .
  • the one or more modules/units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 262 in the video processing device 26.
  • the computer program 262 may be divided into an acquisition unit, a calculation unit, and a stabilization unit. For specific functions of each unit, please refer to the related description in the corresponding embodiment of FIG. 24, which will not be repeated here.
  • the video processing device may include, but is not limited to, a processor 260 and a memory 261.
  • FIG. 26 is only an example of the video processing device 26, and does not constitute a limitation on the video processing device 26. It may include more or less components than shown in the figure, or combine certain components, or be different.
  • the video processing device may also include input and output devices, network access devices, buses, and so on.
  • the so-called processor 260 may be a central processing unit (Central Processing Unit, CPU), other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the memory 261 may be an internal storage unit of the video processing device 26, such as a hard disk or a memory of the video processing device 26.
  • the memory 261 may also be an external storage device of the video processing device 26, such as a plug-in hard disk equipped on the video processing device 26, a smart memory card (Smart Media Card, SMC), and a secure digital (Secure Digital, SD) card, flash card (Flash Card), etc.
  • the memory 261 may also include both an internal storage unit of the video processing device 26 and an external storage device.
  • the memory 261 is used to store the computer program and other programs and data required by the video processing device.
  • the memory 261 can also be used to temporarily store data that has been output or will be output.
  • An embodiment of the present application also provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the foregoing video stabilization method can be implemented.
  • the embodiments of the present application provide a computer program product.
  • the video processing device can realize the above-mentioned video stabilization method when the video processing device is executed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)
  • Television Systems (AREA)

Abstract

本申请提供一种视频稳化方法及装置，涉及视频处理技术领域，能够有效改善视频拍摄过程中由于抖动产生的画面畸变的问题。该方法包括：获取待处理视频中的非首帧视频帧的运动参数；根据所述运动参数确定与所述运动参数对应的几何参数的稳化值，所述几何参数用于描述所述非首帧视频帧的抖动程度；根据所述运动参数与所述几何参数之间的变换关系，将所述几何参数的稳化值转换为稳化运动参数；根据所述稳化运动参数确定所述非首帧视频帧的稳化视频帧；根据所述稳化视频帧，生成所述待处理视频对应的稳化视频。

Description

一种视频稳化方法及装置
本申请要求于2020年03月19日在中国专利局提交的、申请号为202010195539.2、发明名称为“一种视频稳化方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及视频处理技术领域，尤其涉及一种视频稳化方法及装置。
背景技术
随着视频拍摄在人们日常生活和工作中的普及，人们对拍摄的视频质量的需求越来越高。然而在日常拍摄过程中，拍摄的视频往往由于拍摄者的移动，或者拍摄装置处于非静态的环境，导致拍摄的视频出现画面畸变的问题，使得视频质量下降。
技术问题
本申请实施例的目的之一在于：提供一种视频稳化方法及装置，，旨在改善视频拍摄过程中由于抖动产生的画面畸变的问题。
技术解决方案
为解决上述技术问题，本申请实施例采用的技术方案是：
第一方面，本申请提供一种视频稳化方法，包括：获取待处理视频中的非首帧视频帧的运动参数；根据所述运动参数确定与所述运动参数对应的几何参数的稳化值，所述几何参数用于描述所述非首帧视频帧的抖动程度；根据所述运动参数与所述几何参数之间的变换关系，将所述几何参数的稳化值转换为稳化运动参数；根据所述稳化运动参数确定所述非首帧视频帧的稳化视频帧；根据所述稳化视频帧，生成所述待处理视频对应的稳化视频。
采用本申请提供的视频稳化方法，通过计算与每一非首帧视频帧的运动参数对应的几何参数的稳化值，实现了对每一非首帧视频帧抖动程度的直接矫正，在一定程度上消除了每一帧非首帧视频帧由于抖动所产生的画面畸变。因此，保证了得到的稳化视频相比于待处理视频，在一定程度上消除了由于抖动所产生的画面畸变。
可选的，所述非首帧视频帧包括多帧，且所述多帧非首帧视频帧是连续的；相应的，所述根据所述稳化视频帧，生成所述待处理视频对应的稳化视频包括：根据每一帧所述非首帧视频帧各自分别对应的稳化视频帧，生成所述待处理视频对应的稳化视频。
可选的，所述多帧非首帧视频帧与所述待处理视频的首帧视频帧连续，所述根据每一 帧所述非首帧视频帧各自分别对应的稳化视频帧，生成所述待处理视频对应的稳化视频之前，所述方法还包括：根据预设的裁剪比例以及所述首帧视频帧的高和宽，确定所述首帧视频帧的射影变换矩阵；根据所述首帧视频帧的射影变换矩阵对所述首帧视频帧进行裁剪，得到所述首帧视频帧的稳化视频帧；相应的，所述根据所述多帧非首帧视频帧各自分别对应的稳化视频帧，生成所述待处理视频对应的稳化视频，包括：根据所述首帧视频帧的稳化视频帧和每一帧所述非首帧视频帧各自分别对应的稳化视频帧，生成所述待处理视频对应的稳化视频。
可选的，所述根据所述运动参数确定与所述运动参数对应的几何参数的稳化值，包括：根据所述变换关系，将所述运动参数转换为所述几何参数；对所述几何参数进行平滑滤波处理，得到所述几何参数的稳化值。
可选的，所述几何参数包括在第一方向上的第一缩放比例，在第二方向上的第二缩放比例，偏离所述第一方向的偏第一转角度，偏离所述第二方向的第二偏转角度，在所述第一方向上的第一偏移量，在所述第二方向上的第二偏移量，其中，所述第一方向和所述第二方向相互垂直。
可选的，所述对所述几何参数进行平滑滤波处理，得到所述几何参数的稳化值，包括：对所述第一缩放比例和所述第二缩放比例进行平滑滤波处理，得到所述第一缩放比例的稳化值和所述第二缩放比例的稳化值；对所述第一偏转角度和所述第二偏转角度进行平滑滤波处理，得到的所述第一偏转角度的稳化值和所述第二偏转角度的稳化值；对所述第一偏移量进行平滑滤波处理，得到所述第一偏移量的稳化值；对所述第二偏移量进行平滑滤波处理，得到所述第二偏移量的稳化值。
可选的，所述对所述第一缩放比例和所述第二缩放比例进行平滑滤波处理，得到所述第一缩放比例的稳化值和所述第二缩放比例的稳化值，包括：将所述第一缩放比例和所述第二缩放比例带入预设的第一可逆函数中，计算得到所述第一缩放比例和所述第二缩放比例的均值与第一预设均值的第一差异度；利用第一平滑滤波器对所述第一差异度进行平滑滤波处理，得到所述第一差异度的稳化值；将所述第一缩放比例和所述第二缩放比例带入预设的第二可逆函数中，计算得到所述第一缩放比例和所述第二缩放比例之间的第二差异度；利用第二平滑滤波器对所述第二差异度进行平滑滤波处理，得到所述第二差异度的稳化值；利用所述第一差异度的稳化值和所述第二差异度的稳化值对所述第一可你函数和所述第二可逆函数进行线性求解，得到所述第一缩放比例的稳化值和所述第二缩放比例的稳化值。
可选的，所述对所述第一偏转角度和所述第二偏转角度进行平滑滤波处理，得到的所述第一偏转角度的稳化值和所述第二偏转角度的稳化值，包括：将所述第一偏转角度和所 述第二偏转角度带入预设的第三可逆函数中，计算得到所述第一偏转角度和所述第二偏转角度的均值与第三预设均值的第三差异度；利用第三平滑滤波器对所述第三差异度进行平滑滤波处理，得到所述第三差异度的稳化值；将所述第一偏转角度和所述第二偏转角度带入预设的第四可逆函数中，计算得到所述第一偏转角度和所述第二偏转角度之间的第四差异度；利用第四平滑滤波器对所述第四差异度进行平滑滤波处理，得到所述第四差异度的稳化值；利用所述第三差异度和所述第四差异度对所述第三可逆函数和所述第四可逆函数进行线性求解，得到所述第一偏转角度的稳化值和所述第二偏转角度的稳化值。
可选的，所述对所述第一偏移量进行平滑滤波处理，得到所述第一偏移量的稳化值，包括：将所述第一偏移量带入预设的第五可逆函数，计算所述第一偏移量占据所述非首帧视频帧的画面宽度的第一比例；利用第五平滑滤波器对所述第一比例进行平滑滤波处理，得到一个第一稳化值；利用第六平滑滤波器对所述第一比例进行平滑滤波处理，得到第二稳化值；对所述第一稳化值和所述第二稳化值进行加权计算，得到所述第一偏移量的稳化值；其中，所述第五平滑滤波器的状态转移噪声矩阵参数小于所述第六平滑滤波器的状态噪声矩阵参数。
可选的，所述对所述第一稳化值和所述第二稳化值进行加权计算，得到所述第一偏移量的稳化值，包括：根据所述第五平滑滤波器的状态转移噪声矩阵计算与所述第五平滑滤波器对应的加权值；根据与所述第五平滑滤波器对应的加权值，计算与所述第六平滑滤波器对应的加权值；利用与所述第五平滑滤波器对应的加权值和与所述第六平滑滤波器对应的加权值，对所述第一稳化值和所述第二稳化值进行加权计算，得到所述第一偏移量的稳化值。
可选的，所述根据所述第五平滑滤波器的状态转移噪声矩阵计算与所述第五平滑滤波器对应的加权值，包括：利用公式
Figure PCTCN2020130572-appb-000001
计算与所述第五平滑滤波器对应的加权值；
其中，Q 5表示所述第五平滑滤波器的状态转移噪声矩阵，函数
Figure PCTCN2020130572-appb-000002
表示
Figure PCTCN2020130572-appb-000003
的绝对值，dx norm表示所述第一比例，
Figure PCTCN2020130572-appb-000004
表示所述第一稳化值，函数ω()表示差异度计算，0≤w 5≤1。
可选的，所述对所述第二偏移量进行平滑滤波处理，得到所述第二偏移量的稳化值，包括：将所述第二偏移量带入第六可逆函数，计算所述第二偏移量占据所述非首帧视频帧的画面高度的第二比例；利用第七平滑滤波器对所述第二比例进行平滑滤波处理，得到第三稳化值；利用第八平滑滤波器对所述第二比例进行平滑滤波处理，得到第四稳化值；对所述第三稳化值和所述第四稳化值进行加权计算，得到所述第二偏移量的稳化值；其中， 所述第七平滑滤波器的状态转移噪声矩阵参数小于所述第八平滑滤波器的状态噪声矩阵参数。
可选的，所述对所述第三稳化值和所述第四稳化值进行加权计算，得到所述第二偏移量的稳化值，包括：利用所述第七平滑滤波器的状态转移噪声矩阵计算与所述第七平滑滤波器对应的加权值；根据与所述第七平滑滤波器对应的加权值，计算与所述第八平滑滤波器对应的加权值；利用与所述第七平滑滤波器对应的加权值和与所述第八平滑滤波器对应的加权值，对所述第三稳化值和所述第四稳化值进行加权计算，得到所述第二偏移量的稳化值。
可选的，所述利用所述第七平滑滤波器的状态转移噪声矩阵计算与所述第七平滑滤波器对应的加权值，包括：利用公式
Figure PCTCN2020130572-appb-000005
计算与所述第七平滑滤波器对应的加权值w 7；其中，Q 7表示所述第七平滑滤波器的状态转移噪声矩阵，函数
Figure PCTCN2020130572-appb-000006
表示
Figure PCTCN2020130572-appb-000007
的绝对值，dy norm表示所述第二比例，表示所述第三稳化值，0≤w 7≤1。
可选的，所述根据所述稳化运动参数确定所述非首帧视频帧的稳化视频帧，包括：根据所述稳化运动参数计算所述非首帧视频帧的射影变换矩阵；利用所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到所述非首帧视频帧的稳化视频帧。
可选的，所述利用所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到所述非首帧视频帧的稳化视频帧，包括：利用所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到稳化子视频帧；若所述稳化子视频帧的尺寸大小与所述非首帧视频帧的尺寸大小相同，则确定所述稳化子视频帧为所述非首帧视频帧的稳化视频帧。
可选的，所述利用所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到稳化子视频帧之后，所述方法还包括：若所述稳化子视频帧的尺寸大小与所述非首帧视频帧的尺寸大小不同，则对所述射影变换矩阵进行调整；根据调整后的所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到所述非首帧视频帧的稳化视频帧。
第二方面，本申请提供一种视频处理装置，包括：获取单元，用于获取运动参数，所述运动参数为待处理视频中非首帧非首帧视频帧的运动参数；计算单元，用于根据所述运动参数计算与所述运动参数对应的几何参数的稳化值，所述几何参数用于描述所述非首帧视频帧的抖动程度；并根据所述运动参数与所述几何参数之间的变换关系，将所述几何参数的稳化值转换为稳化运动参数；稳化单元，用于根据所述稳化运动参数确定所述非首帧视频帧的稳化视频帧。
第三方面，本申请提供一种视频处理设备，处理器、存储器以及存储在所述存储器中 并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如第一方面或第一方面的任意可选方式所述的方法。
第四单元，本申请提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现如第一方面或第一方面的任意可选方式所述的方法。
第五方面，本申请实施例提供了一种计算机程序产品，当计算机程序产品在视频处理设备上运行时，使得视频处理设备执行上述第一方面所述的视频质量评估方法的步骤。
可以理解的是，上述第二方面至第五方面的有益效果可以参见上述第一方面中的相关描述，在此不再赘述。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种视频稳化原理示意图；
图2是本申请实施例提供的一种视频稳化方法的一个实施例的流程示意图；
图3是本申请实施例提供的一种几何参数与运动参数之间的变换关系的示意图；
图4是本申请实施例提供的测试视频1的原始视频帧和对应的稳化视频帧的对比示意图；
图5是本申请实施例提供的测试视频1的运动参数中的参数a和参数a的稳化值的变化曲线对比示意图；
图6是本申请实施例提供的测试视频1的运动参数中的参数b和参数b的稳化值的变化曲线对比示意图；
图7是本申请实施例提供的测试视频1的运动参数中的参数c和参数c的稳化值的变化曲线对比示意图；
图8是本申请实施例提供的测试视频1的运动参数中的参数d和参数d的稳化值的变化曲线对比示意图；
图9是本申请实施例提供的测试视频1的运动参数中的参数e和参数e的稳化值的变化曲线对比示意图；
图10是本申请实施例提供的测试视频1的运动参数中的参数f和参数f的稳化值的变化曲线对比示意图；
图11是本申请实施例提供的测试视频2的原始视频帧和对应的稳化视频帧的对比示意图；
图12是本申请实施例提供的测试视频2的运动参数中的参数a和参数a的稳化值的变化曲线对比示意图；
图13是本申请实施例提供的测试视频2的运动参数中的参数b和参数b的稳化值的变化曲线对比示意图；
图14是本申请实施例提供的测试视频2的运动参数中的参数c和参数c的稳化值的变化曲线对比示意图；
图15是本申请实施例提供的测试视频2的运动参数中的参数d和参数d的稳化值的变化曲线对比示意图；
图16是本申请实施例提供的测试视频2的运动参数中的参数e和参数e的稳化值的变化曲线对比示意图；
图17是本申请实施例提供的测试视频2的运动参数中的参数f和参数f的稳化值的变化曲线对比示意图；
图18是本申请实施例提供的测试视频3的原始视频帧和对应的稳化视频帧的对比示意图；
图19是本申请实施例提供的测试视频3的运动参数中的参数a和参数a的稳化值的变化曲线对比示意图；
图20是本申请实施例提供的测试视频3的运动参数中的参数b和参数b的稳化值的变化曲线对比示意图；
图21是本申请实施例提供的测试视频3的运动参数中的参数c和参数c的稳化值的变化曲线对比示意图；
图22是本申请实施例提供的测试视频3的运动参数中的参数d和参数d的稳化值的变化曲线对比示意图；
图23是本申请实施例提供的测试视频3的运动参数中的参数e和参数e的稳化值的变化曲线对比示意图；
图24是本申请实施例提供的测试视频3的运动参数中的参数f和参数f的稳化值的变化曲线对比示意图；
图25是本申请实施例提供的一种视频稳化装置的结构示意图；
图26是本申请实施例提供的一种视频处理设备的结构示意图；
图27是本申请实施例提供的一种视频处理系统的系统架构示意图。
本发明的实施方式
以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、技术之类的具体细节，以便透彻理解本申请实施例。然而，本领域的技术人员应当清楚，在没有这些具体细节的其它实施例中也可以实现本申请。在其它情况中，省略对众所周知的系统、装置、电路以及方法的详细说明，以免不必要的细节妨碍本申请的描述。
应当理解，在本申请说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。另外，在本申请说明书和所附权利要求书的描述中，术语“第一”、“第二”、“第三”等仅用于区分描述，而不能理解为指示或暗示相对重要性。
还应当理解，在本申请说明书中描述的参考“一个实施例”或“一些实施例”等意味着在本申请的一个或多个实施例中包括结合该实施例描述的特定特征、结构或特点。由此，在本说明书中的不同之处出现的语句“在一个实施例中”、“在一些实施例中”、“在其他一些实施例中”、“在另外一些实施例中”等不是必然都参考相同的实施例，而是意味着“一个或多个但不是所有的实施例”，除非是以其他方式另外特别强调。术语“包括”、“包含”、“具有”及它们的变形都意味着“包括但不限于”，除非是以其他方式另外特别强调。
在说明本申请提供视频稳化方法之前，首先对结合图1对本申请所采用的稳化原理以及视频拍摄中的相关概念进行示例性的说明。
为了便于描述，在本申请实施例中，视频处理设备拍摄到的待处理视频中的视频帧用F表示。如图1所示，待处理视频的第1个视频帧(即首帧视频帧)表示为F 1，第2个视频帧表示为F 2、第3个视频帧表示为F 3、第4个视频帧表示为F 4，以此类推。
相邻两个视频帧之间存在射影变换关系，该射影变换关系可以通过运动参数
Figure PCTCN2020130572-appb-000008
来表示。即运动参数M用于表示相邻两个视频帧之间的射影变换关系。
在本申请中，假设待处理视频中的第t(t＞1)个视频帧F t的运动参数表示为M t。那么，以F t和F t的前一个视频帧F t-1为例。F t的图像内容和F t-1的图像内容相互之间可以通过基于运动参数M t的二维平面射影变换近似表示，即F t≈f(F t-1,M t)，
Figure PCTCN2020130572-appb-000009
其中，函数f表示射影变换。
例如，假设(x t-1 y t-1 1)为F t-1中坐标点(x t-1 y t-1)的齐次坐标表示，(x t y t 1)为F t中与坐标点(x t-1 y t-1)对应的坐标点(x t y t)的齐次坐标表示。那么，当
Figure PCTCN2020130572-appb-000010
时，则有
Figure PCTCN2020130572-appb-000011
其中，M可以基于运动估计计算所得。常规的运动估计法包括两种，一种是视频处理设备直接基于F t和F t-1的图像内容进行运动估计，得到M t。另一种是视频处理设备通过获取辅助传感器(例如陀螺仪)的数据，对F t和F t-1的图像内容进行运动估计，得到M t。运动估计的具体实施方式，可以参考现有的运动估计的实施方式，对此，本申请不作介绍。
在本申请实施例中，当对待处理视频进行稳化后，得到的每个视频帧的稳化视频帧用S表示。如图1所示，F 1的稳化视频帧表示为S 1，F 2的稳化视频帧表示为S 2，F 3的稳化视频帧表示为S 3，F 4的稳化视频帧表示为S 4，以此类推。
相应的，相邻两个稳化视频帧之间也存在射影变换关系。同样的，相邻两个稳化视频帧之间的射影变换关系也可以通过稳化运动参数K来表示。例如，F t的稳化视频帧S t的图像内容和F t-1的稳化视频帧S t-1的图像内容相互之间，可以通过基于稳化运动参数K t二维平面射影变换近似表示，即S t≈f(S t-1,K t)，
Figure PCTCN2020130572-appb-000012
也就是说，K t是对M t经过稳化后得到的理想匀速运动参数。
假设，如图1所示，F t的稳化视频帧S t，是基于一个射影变换矩阵(为了便于描述，本申请中用R t来表示)对F t进行裁剪所得，那么就有S t≈f(F t,R t)。
以图1中的F 1、F 2、S 1、S 2为例，由于存在F 2≈f(F 1,M 2)，
Figure PCTCN2020130572-appb-000013
S 2≈f(S 1,K 2)，S 1≈f(F 1,R 1)，因此，基于射影变换的连乘法则，可以得到：
Figure PCTCN2020130572-appb-000014
进一步的，基于数学归纳法，可以得到如下关系式：
Figure PCTCN2020130572-appb-000015
因此，可以得到
Figure PCTCN2020130572-appb-000016
可以看出，在本申请实施例中，当计算F t的稳化视频帧S t时，可以通过求解F t和F t-1之间的运动参数M t的稳化运动参数K t来求解射影变换矩阵R t，进而通过R t获取F t的S t。
下面通过具体实施例，对本申请提供的视频稳化方法进行示例性的说明。
本申请提供的视频稳化方法执行主体为视频处理设备。该视频处理设备可以是终端设备(包括但不限于智能手机、平板电脑、可穿戴设备等移动终端，还可以是各种应用场景下的摄像机、机器人等)，也可以是服务器。即本申请提供的视频稳化方法可以由终端设备执行，也可以由服务器执行。
示例性的，如图27所示，为本申请提供的一种视频处理系统，包括至少一个终端设备和服务器。终端设备获取到待处理视频后，可以将待处理视频发送给服务器。服务器通过执行本申请提供的视频稳化方法，对该待处理视频进行稳化，得到该待处理视频的稳化视频。之后服务器将稳化视频发送给终端设备。
或者，终端设备也可以在获取到处理视频后，通过执行本申请提供的视频稳化方法，对该待处理视频进行稳化，得到该待处理视频的稳化视频。
请参见图2，图2是本申请一个实施例提供的一种视频稳化方法的示意流程图。如图2所示的视频稳化方法可包括：
S201，获取待处理视频中的非首帧视频帧的运动参数。
在一个示例中，待处理视频中除第一帧(即首帧)视频帧以外的任意一个视频帧均为非首帧视频帧。例如，该待处理视频一共包括10帧视频帧，第2帧到第10帧均为非首帧视频帧。
视频处理设备可以在获取到非首帧视频帧和播放时间早于该非首帧视频的前一个视频帧后，进行运动估计，得到该非首帧视频帧的运动参数。例如，F t为非首帧视频帧，那么视频处理设备可以对F t和F t-1的图像内容进行运动估计，得到F t的运动参数M t。
S202，根据所述运动参数确定与所述运动参数对应的几何参数的稳化值。
在本申请实施例中，可以通过定义几何参数来描述视频帧的抖动程度，即几何参数可以理解为用于描述视频帧的抖动程度的参数。例如，几何参数可以包括视频帧的缩放程度、偏转程度和/或偏移程度等。需要说明的是，视频帧的抖动程度可以理解为该视频帧与相邻 视频帧或其他视频帧之间相对而言的抖动程度，即该视频帧与相邻视频帧或其他视频帧之间相对而言所发生的画面整体偏移程度、缩放程度、偏转程度。
比如，对于偏移程度来讲，非首帧视频帧与其相邻视频帧之间所发生的画面整体的位移越大，则非首帧视频帧的抖动程度越大，反之，非首帧视频帧与其相邻视频帧之间所发生的画面整体的位移越小，则非首帧视频帧的抖动程度越小。可以理解的是，若非首帧视频帧与其相邻视频帧之间所发生的画面整体的位移大于预设偏移阈值时，可以认为非首帧视频帧发生了抖动，若非首帧视频帧与其相邻视频帧之间所发生整体的位移小于或等于该预设偏移阈值时，可以认为非首帧视频帧实现了稳化。
对于缩放程度来讲，非首帧视频帧与其相邻视频帧之间所发生的画面整体的缩放程度越大，则非首帧视频帧的抖动程度越大，反之，非首帧视频帧与其相邻视频帧之间所发生的画面整体的缩放程度越小，则非首帧视频帧的抖动程度越小。可以理解的是，若非首帧视频帧与其相邻视频帧之间所发生的画面整体的缩放程度大于预设缩放阈值时，可以认为非首帧视频帧发生了抖动，若非首帧视频帧与其相邻视频帧之间所发生整体的缩放程度小于或等于该预设偏缩放值时，可以认为非首帧视频帧实现了稳化。
对于偏转程度来讲，非首帧视频帧与其相邻视频帧之间所发生的画面整体的偏转越大，则非首帧视频帧的抖动程度越大，反之，非首帧视频帧与其相邻视频帧之间所发生的画面整体的偏转越小，则非首帧视频帧的抖动程度越小。可以理解的是，若非首帧视频帧与其相邻视频帧之间所发生的画面整体的偏转大于预设偏转角阈值时，可以认为非首帧视频帧发生了抖动，若非首帧视频帧与其相邻视频帧之间所发生整体的偏转小于或等于该预设偏转角阈值时，可以认为非首帧视频帧实现了稳化。
在本申请实施例中，通过建立几何参数和运动参数之间的变换关系，实现几何参数与运动参数之间的相互转换。
示例性的，由于待处理视频是由一帧一帧的二维视频帧组成。因此，每一帧非首帧视频帧的几何参数可以在两个相互垂直的方向(假设为第一方向和第二方向)上进行定义。例如，几何参数可以包括在第一方向上的第一缩放比例V x，在第二方向上的第二缩放比例V y，偏离所述第一方向的第一偏转角度θ x，偏离所述第二方向的第二偏转角度θ y，在所述第一方向上的第一偏移量dx和/或在所述第二方向上的第二偏移量dy。可以理解的是，V x和V y可以描述视频帧的缩放程度，θ x和θ y可以描述视频帧的偏转程度，dx和dy可以描述视频帧的偏移程度。
下面以对几何参数包括V x、V y、θ x、θ y、dx、dy为例，结合图3对几何参数V x、V y、θ x、θ y、dx、dy与M之间的变换关系进行示例性的说明。
如图3所示，建立相互垂直的第一方向的坐标轴(x轴)和第二方向的坐标轴(y轴)，x轴和y轴的原点为O点，形成坐标系O x-y。在本申请实施例中，以O x-y作为预设的基准坐标，视频处理设备以视频帧的实际坐标系与坐标系O x-y的差异度，来衡量该视频帧的抖动程度。
如图3所示，x轴按照第一偏转角度θ x旋转，并基于第一缩放比例V x伸缩后形成x1轴。y轴按照第二偏转角度θ y旋转，并基于第二缩放比例V y伸缩后形成y1轴。x1轴沿着y1轴方向偏移第二偏移量dy后形成x2轴，y1沿着x1轴方向偏移第一偏移量dx后形成y2轴。x2轴与y2轴的交点B为x2轴与y2轴的坐标原点。假设原点B、x2轴与y2轴构成的坐标系B x2-y2为视频帧的实际坐标系。
可以理解为，视频帧经过抖动后，在第一方向上伸缩V x，在第二方向上的伸缩V y，偏离第一方向θ x，偏离第二方向θ y，在第一方向上偏移dx，并在第二方向上的偏移dy，从基准坐标系O x-y抖动到坐标系B x2-y2。
针对基准坐标系O x-y和坐标系B x2-y2进行仿射变换。首先，如图3所示，标注点A、B、C、D、E、F、G、H、I、J、L并进行连线，使得∠PFE＝∠PDO＝∠GEO＝∠PGE＝∠PHI＝∠PIO＝∠PJL＝∠IKO＝90°，且
Figure PCTCN2020130572-appb-000017
其中，P点在O x-y中的坐标假设为(x old,y old)，在B x2-y2中的坐标为(x new,y new)。那么，基于几何证明可以得到：
由于
Figure PCTCN2020130572-appb-000018
因此可以得到x new＝V x·cos(θ x)·x old-V x·sin(θ x)·y old+dx。相应的，由于
Figure PCTCN2020130572-appb-000019
Figure PCTCN2020130572-appb-000020
因此可以得到
Figure PCTCN2020130572-appb-000021
结合M中参数a、b、c、d、e、f，构建如下方程组：
Figure PCTCN2020130572-appb-000022
可以得到一个六元方程组：
a＝V x·cos(θ x)；b＝-V x·sin(θ x)；c＝dx；d＝V y·sin(θ y)；e＝V y·cos(θ y)；f＝dy。
在该示例中，以该六元方程组作为M与几何参数之间的变换关系，实现M与几何参数的相互变换。即通过该几何变换关系，可以基于运动参数直接获得能够反应非首帧视频帧的抖动程度的几何参数。
在本申请实施例中，视频处理设备可以基于上述变换关系(即M与几何参数之间的变换关系)，将非首帧视频帧的运动参数转换为对应的几何参数。需要说明的是，不同的非首帧视频帧的运动参数M中的各个参数值(即a、b、c、d、e、f)可能不同，因此，将不同的M代入上述六元方程组后，得到的几何参数也不同。
视频处理设备得到非首帧视频帧的运动参数所对应的几何参数后，即可对该几何参数进行平滑滤波处理，得到该几何参数的稳化值。
其中，视频处理设备可以基于卡尔曼滤波器对几何参数进行平滑滤波处理，以消除几何参数的噪声，减少突变，使得几何参数的稳化值在非首帧视频帧的相邻上一个视频帧的基础上稳定变化。
在本申请实施例中，由于几何参数是能够反映非首帧视频帧的抖动程度的参数，因此，通过对几何参数进行平滑滤波处理，可以消除几何参数的抖动，实现对几何参数的稳化，能够进一步消除由于抖动所产生的画面畸变，保证非首帧视频帧的稳化效果。
其中，所谓非首帧视频帧的稳化是指使得非首帧视频帧呈现相对稳定的画面，在一定程度上消除由于抖动所产生的画面畸变。
在一个示例中，视频处理设备可以直接利用平滑滤波器分别对每个几何参数进行平滑滤波处理，得到几何参数的稳化值。其中，平滑滤波器可以是卡尔曼(Kalman)滤波器。
在另一个示例中，视频处理设备可以将于非首帧视频帧对应的几何参数分四组进行滤波。示例性的，四组滤波分别为：
(1)对V x和V y进行平滑滤波处理，得到V x的稳化值V x K和V y的稳化值V y K。
(2)对θ x和θ y进行平滑滤波处理，得到θ x的稳化值θ x K和θ y的稳化值θ y K。
(3)对dx进行平滑滤波处理，得到dx的稳化值dx K；
(4)对dy进行平滑滤波处理，得到dy的稳化值dy K。
针对第(1)组，当V x和V y的值差异较大时，非首帧视频帧的画面会出现高宽不等比缩放的畸变。而在该示例中，通过对对应的V x和V y进行组合滤波，可以有效的改善非首帧视频帧在不同方向上基于不同缩放比例的缩放造成的畸变。
示例性的，对于第(1)组，视频处理设备可以通过如下步骤S01-S05实现：
S01，将V x和V y带入预设的第一可逆函数g 1(V x,V y,V c)中，计算得到V x和V y的均值与第一预设均值V c的第一差异度G 1。
其中，V c可以取值为1，也可以取值为与非首帧视频帧相邻的上一个视频帧F t-1的V x和V y的均值。
S02，利用第一平滑滤波器对G 1进行平滑滤波处理，得到G 1的稳化值G 1 K。
S03，将所述几何参数中的V x和V y的值带入预设的第二可逆函数g 2(V x,V y)中，计算得到所述几何参数中的V x和V y之间的第二差异度G 2。
S04，利用第二平滑滤波器对G 2进行平滑滤波处理，得到G 2的稳化值G 2 K。
其中，第一平滑滤波器和第二平滑滤波器可以是参数相同的卡尔曼滤波器。
S05，利用G 1 K和G 2 K对g 1(V x,V y,V c)和g 2(V x,V y)进行线性求解，得到V x K和V y K。
由于g 1(V x,V y,V c)和g 2(V x,V y)是可逆方程，因此将G 1 K和G 2 K带入后，可以通过构建如下方程组：
Figure PCTCN2020130572-appb-000023
并对该方程组进行线性求解，几何得到对应的V x K和V y K。基于S01-S05可知，视频处理设备通过对V x和V y的值之间的差异进行稳化，避免了求得的V x K和V y K出现较大的差异，从而能够有效改善非首帧视频帧在不同方向上基于不同缩放比例的缩放造成的畸变。
针对第(2)组，当θ x和θ y的值差异较大时，非首帧视频帧的画面会出现坐标轴不等比扭曲的畸变。而在该示例中，通过对θ x和θ y的值进行组合滤波，可以有效的改善非首帧 视频帧的画面在坐标轴不等比旋转时造成的畸变。
示例性的，对于第(2)组，视频处理设备可以通过如下步骤S06-S10实现：
S06，将θ x和θ y带入预设的第三可逆函数g 3(θ x,θ y,θ c)中，计算得到θ x和θ y的均值与预设均值θ c的第三差异度G 3。
其中，θ c可以取值为0，也可以取值为与非首帧视频帧相邻的前一帧视频帧的θ x和θ y的均值。
S07，利用第三平滑滤波器对G 3进行平滑滤波处理，得到G 3的稳化值G 3 K。
S08，将θ x和θ y带入预设的第四可逆函数g 4(θ x,θ y)中，计算得到θ x和θ y之间的第四差异度G 4。
S09，利用第四平滑滤波器对G 4进行平滑滤波处理，得到G 4的稳化值G 4 K。
其中，第三平滑滤波器和第四平滑滤波器可以是参数相同的卡尔曼滤波器。
S10，利用G 3 K和G 4 K对g 3(θ x,θ y,θ c)和g 4(θ x,θ y)进行线性求解，得到θ x K和θ y K。
由于g 3(θ x,θ y,θ c)和g 4(θ x,θ y)是可逆方程，因此将G 3 K和G 4 K带入后，可以通过构建如下方程组：
Figure PCTCN2020130572-appb-000024
并对该方程组进行线性求解，几何得到对应的θ x K和θ y K。基于S06-S110可知，视频处理设备通过对θ x和θ y的值之间的差异进行稳化，避免了求得的θ x K和θ y K出现较大的差异，从而能够有效改善视频帧的画面在坐标轴不等比旋转时造成的畸变。
值得说明的是，由于V x、V y、θ x、θ y表征的是视频帧的画面的旋转和缩放，与视频帧的尺寸无关，从而直接基于平滑滤波器进行稳化。而dx、dy是位移量，单位是像素，与视频帧的尺寸相关，因此在对dx、dy进行稳化前，需要先进行归一化操作。
示例性的，对于第(3)组，视频处理设备可以通过如下步骤S11-S14实现：
S11，将dx带入预设的第五可逆函数g 5(dx,norm_x)，计算dx占据所述非首帧视频帧的画面宽度norm_x的第一比例dx norm。
其中，dx norm即为dx的归一化值。
S12，利用第五平滑滤波器对dx norm进行平滑滤波处理，得到第一稳化值
Figure PCTCN2020130572-appb-000025
S13，利用第六平滑滤波器对dx norm进行平滑滤波处理，得到第二稳化值
Figure PCTCN2020130572-appb-000026
其中，在一种实现方式中，第五平滑滤波器和第六平滑滤波器均可以为卡尔曼滤波器。且第五平滑滤波器和第六平滑滤波器的转换与观测噪声矩阵参数完全相同，但第五平滑滤波器的状态转移噪声矩阵参数小于第六平滑滤波器的状态噪声矩阵参数。
S14，对
Figure PCTCN2020130572-appb-000027
和
Figure PCTCN2020130572-appb-000028
进行加权计算，得到dx K。
示例性的，视频处理设备可以基于第五平滑滤波器和第六平滑滤波器的权重来控制dx是否被消除。其中，第五平滑滤波器可以对应需要被消除的水平抖动。例如，当运动幅度较小，即dx norm较小时，即可认为dx是需要被消除的水平抖动。第六平滑滤波器可以对应用户刻意的运动。例如，当运动幅度较大，即dx norm较大时，大概率上会是用户刻意运动造成的，此时即可认为dx是不需要被消除的水平抖动。
示例性的，在上述S14中，视频处理设备可以根据所述第五平滑滤波器的状态转移噪声矩阵Q 5计算与所述第五平滑滤波器对应的加权值w 5；根据w 5，计算与所述第六平滑滤波器对应的加权值w 6；利用w 5和w 6，对
Figure PCTCN2020130572-appb-000029
和
Figure PCTCN2020130572-appb-000030
进行加权计算，得到dx K。
在一个示例中，视频处理设备具体可以利用如下公式计算w 5：
Figure PCTCN2020130572-appb-000031
其中，函数
Figure PCTCN2020130572-appb-000032
表示
Figure PCTCN2020130572-appb-000033
的绝对值，函数ω()表示差异度计算，0≤w 5≤1。
然后利用如下公式w 6＝1-w 5计算w 6：
可以看出，当dx norm较小时，w 5较大，当dx norm较大时，w 6较大。
最后，将加权值w 5和w 6带入利用公式
Figure PCTCN2020130572-appb-000034
计算dx K。
即dx K是dx经过归一化后的加权稳化值。
示例性的，对于第(4)组，视频处理设备可以通过如下步骤S15-S18实现：
S15，将dy带入第六可逆函数g 6(dy,norm_y)，计算dy占据所述非首帧视频帧的画 面高度norm_y的第二比例dy norm。
S16，利用第七平滑滤波器对dy norm进行平滑滤波处理，得到第三稳化值
Figure PCTCN2020130572-appb-000035
S17，利用第八平滑滤波器对dy norm进行平滑滤波处理，得到第四稳化值
Figure PCTCN2020130572-appb-000036
其中，第七平滑滤波器和第八平滑滤波器均可以为卡尔曼滤波器。且第五平滑滤波器和第八平滑滤波器的转换与观测噪声矩阵参数完全相同，但第七平滑滤波器的状态转移噪声矩阵参数小于第八平滑滤波器的状态噪声矩阵参数。
S18，对
Figure PCTCN2020130572-appb-000037
和
Figure PCTCN2020130572-appb-000038
进行加权计算，得到dy K。
示例性的，视频处理设备可以基于第七平滑滤波器和第八平滑滤波器的权重来控制dy是否被消除。其中，第七平滑滤波器可以对应需要被消除的垂直抖动，例如，当运动幅度较小，即dy norm较小时，即可认为dy是需要被消除的垂直抖动。第八平滑滤波器可以对应用户刻意的运动，例如，当运动幅度较大，即dy norm较大时，大概率上会是用户刻意运动造成的，此时即可认为dy是不需要被消除的垂直抖动。
示例性的，在上述S18中，视频处理设备可以利用所述第七平滑滤波器的状态转移噪声矩阵Q 7计算与所述第七平滑滤波器对应的加权值w 7；然后根据w 7，计算与所述第八平滑滤波器对应的加权值w 8；最后利用w 7和w 8，对
Figure PCTCN2020130572-appb-000039
和
Figure PCTCN2020130572-appb-000040
进行加权计算，得到dy K。
在一个示例中，视频处理设备具体可以利用公
Figure PCTCN2020130572-appb-000041
计算w 7。其中，函数
Figure PCTCN2020130572-appb-000042
表示
Figure PCTCN2020130572-appb-000043
的绝对值，0≤w 7≤1。
然后利用公式w 8＝1-w 7计算w 8。
可以看出，当dy norm较小时，w 7较大，当dy norm较大时，w 8较大。
最后，将w 7和w 8带入公式
Figure PCTCN2020130572-appb-000044
计算dy K。
即dy K是dy经过归一化后的加权稳化值。
S203，根据运动参数与对应的几何参数之间的变换关系和所述几何参数的稳化值，计算得到运动参数的稳化运动参数。
例如，当视频处理设备得到各个几何参数的稳化值后，通过将各个几何参数的稳化值重新带入上述六元方程组中进行线性求解到运动参数的稳化运动参数。例如，将M t对应 的几何参数的稳化值，带入上述六元方程组进行线性求解后，得到M t中的a的稳化值a K、b的稳化值b K、c的稳化值c K、d的稳化值d K、e的稳化值e K、以及f的稳化值f K。
基于a K、b K、c K、d K、e K、以及f K可以得到M t的稳化运动参数
Figure PCTCN2020130572-appb-000045
S204，根据稳化运动参数确定所述非首帧视频帧的稳化视频帧。
示例性的，参考如图1所示示例中的推导过程，视频处理设备可以根据所述稳化运动参数计算所述非首帧视频帧的射影变换矩阵。
例如，非首帧视频帧为F t，运动参数为M t，那么运动参数的稳化运动参数为K t。基于图1所示实施例可知，视频处理设备可以利用公式
Figure PCTCN2020130572-appb-000046
计算得到F t的射影变换矩阵R t。
当t＝2时，
Figure PCTCN2020130572-appb-000047
即当计算F 2的R 2时，需要用到首帧F 1的R 1。而待处理视频的首帧不存在运动参数，因此视频处理设备可以将中心裁剪矩阵
Figure PCTCN2020130572-appb-000048
作为F 1的R 1。其中，r表示预设的裁剪比例，W表示F 1的宽，H表示F 1的高。
即对于首帧F 1来说，
Figure PCTCN2020130572-appb-000049
得到目标射频帧的射影变换矩阵后，即可利用所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到所述非首帧视频帧的稳化视频帧。
基于射影变换的原理可知，采用射影变换矩阵对视频帧进行裁剪，是一个基于射影变换矩阵从视频帧中提取像素点的过程。由于视频在录制过程中，可能由于视频采集装置处于运动状态，导致相邻的两帧视频帧中，播放时间较早的前一个视频帧存在的部分图像内容，在播放时间较晚的后一帧视频帧中不存在。因此采用计算所得的射影变换矩阵可能从非首帧视频帧中获取部分像素的图像，即出现图像越界的情况。
那么，视频处理设备在利用所述射影变换矩阵对所述非首帧视频帧的裁剪获得的稳化视频帧时，可以先利用射影变换矩阵对所述非首帧视频帧进行裁剪，得到稳化子视频帧。 若所述稳化子视频帧的尺寸大小与所述非首帧视频帧的尺寸大小相同，则确定所述稳化子视频帧为所述非首帧视频帧的稳化视频帧。稳化子视频帧的尺寸大小与所述非首帧视频帧的尺寸大小相同，说明射影变换矩阵未导致稳化视频帧出现图像越界的问题。
若所述稳化子视频帧的尺寸大小与所述非首帧视频帧的尺寸大小不同，则表明计算的射影变换矩阵对非首帧视频帧进行裁剪后，得到的稳化视频帧图像越界。那么，视频处理设备可以对射影变换矩阵进行调整。然后根据调整后的所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到所述非首帧视频帧的稳化视频帧。
示例性的，假设视频帧中第一方向的长度表示视频帧的宽度，与第一方向垂直的两条边分别为左边和右边。视频帧中第二方向的长度表示视频帧的高度，与第二方向垂直的两条边分别为上边和下边。下面以F t为非首帧视频帧为例。
当所述稳化子视频帧的宽度小于F t的宽度时，终端先确定是哪一边越界。例如，若R t所指示的稳化子视频帧的右边上的像素点在F t中不存在对应的像素点，即无法在F t中提取到对应的图像内容，则表示稳化子视频帧的右边越界。视频处理设备即可调整R t，使得根据调整后的R t裁剪所得稳化子视频帧的右边与F t的右边对齐。也就是使得调整后的根据调整后的R t裁剪所得稳化子视频帧的右边上的像素点与F t的左边上的像素点对应。同样的，若R t所指示的稳化子视频帧的左边上的像素点在F t中不存在对应的像素点，即无法在F t中提取到对应的图像内容，则表示稳化子视频帧左边越界。视频处理设备即可调整R t，使得根据调整后的R t裁剪所得稳化子视频帧的左边与F t的左边对齐。也就是使得调整后的根据调整后的R t裁剪所得稳化子视频帧的右边上的像素点与F t的左边上的像素点对应。
当所述稳化子视频帧的宽度大于F t的宽度时，视频处理设备则可以通过调整几何参数的稳化值中第一方向和第二方向的缩放比例，来调整R t，使得根据调整后的R t裁剪所得的稳化子视频帧的宽度与F t的宽度相同。
相应的，当所述稳化子视频帧的高度小于F t的高度时，若R t所指示的稳化子视频帧的上边上的像素点在F t中不存在对应的像素点，即无法在F t中提取到对应的图像内容，则表示稳化子视频的上边越界。视频处理设备即可调整R t，使得根据调整后的R t裁剪所得 稳化子视频帧的上边与F t的上边对齐。也就是使得调整后的根据调整后的R t裁剪所得稳化子视频帧的上边上的像素点与F t的上边上的像素点对应。同样的，若R t所指示的稳化子视频帧的下边上的像素点在F t中不存在对应的像素点，即无法在F t中提取到对应的图像内容，则表示稳化子视频的下边越界。视频处理设备即可调整R t，使得根据调整后的R t裁剪所得稳化子视频帧的下边与F t的下边对齐。也就是使得调整后的根据调整后的R t裁剪所得稳化子视频帧的下边上的像素点与F t中的边上的像素点对应。
当所述稳化子视频帧的高度大于F t的高度时，视频处理设备则可以通过调整几何参数的稳化值中第一方向和第二方向的缩放比例，来调整R t，使得根据调整后的R t裁剪所得稳化子视频帧的高度与F t的高度相同。
采用本申请提供的视频稳化方法，通过计算与运动参数对应的几何参数的稳化值，得到运动参数的稳化运动参数，可以实现对非首帧视频帧抖动程度的直接矫正，保证了非首帧视频帧的稳化效果，能够有效改善非首帧视频帧由于抖动所产生的画面的畸变。
在本实施例中，视频处理设备可以根据实际需要对待处理视频中的一帧非首帧视频帧进行处理，也可以对待处理视频帧中的多帧非首帧视频帧进行处理，在本实施例中并不对非首帧视频帧的数量进行限定。
S205，根据所述非首帧视频帧的稳化视频帧，生成所述待处理视频对应的稳化视频。
在本申请实施例中，稳化视频由视频处理设备对待处理视频中的至少一帧视频帧进行稳化处理后得到的稳化视频帧构成。该至少一帧视频帧可以是连续的非首帧视频帧，也可以是包括首帧视频帧在内的连续视频帧。
也就是说，视频处理设备可以对整个待处理视频进行稳化处理，也可以对由待处理视频中的至少一个连续的视频帧构成的视频片段进行稳化处理。比如，待处理视频的时长为1分钟，若用户需要对播放时间为第10s至第20s之间的全部视频帧进行稳化处理，则可以将播放时间为第10s至第20s之间的全部视频帧作所构成的视频片段作为需要进行视频稳化的视频片段。或者，若用户需要对时长为1分钟的待处理视频整个进行稳化处理，则可以直接对该待处理视频进行稳化处理。
可以理解的是，如果视频处理设备需要对由连续的多帧非首帧视频帧构成的视频片段进行稳化，那么视频处理设备可以基于上述步骤S201-S204对每一帧非首帧视频帧进行稳化，得到每一帧非首帧视频帧的稳化视频帧。相应的，在上述S205中，视频处理设备可以 根据每一帧所述非首帧视频帧各自分别对应的稳化视频帧，生成所述待处理视频对应的稳化视频。
如果视频处理设备需要对整个待处理视频或者包含首帧视频帧在内的视频片段进行稳化，那么视频处理设备可以基于上述步骤S201-S204对待处理视频或者视频片段中的每一帧非首帧视频帧进行稳化，得到每一帧非首帧视频帧的稳化视频帧。相应的，在上述S205中，视频处理设备可以根据所述首帧视频帧帧和每一帧所述非首帧视频帧各自分别对应的稳化视频帧，生成所述待处理视频对应的稳化视频。
在一个示例中，由于首帧视频帧是待处理视频中第一个拍摄到的图像，后续拍摄到的非首帧视频帧存在的抖动，可以理解为以首帧视频帧的画面作为基准，相对于首帧视频帧的画面所发生的画面整体的抖动。因此，视频处理设备可以直接将首帧视频帧作为待处理视频对应的稳化视频的首帧视频帧。并将该首帧视频帧和每一帧非首帧视频帧各自分别对应的稳化视频帧确定为稳化视频。
在另一个示例中，视频处理设备也可以对首帧视频帧进行稳化。例如，视频处理设备在利用预设的裁剪比例以及所述首帧视频帧的高和宽，确定所述首帧视频帧的射影变换矩阵(即上述计算的中心裁剪矩阵
Figure PCTCN2020130572-appb-000050
)后，也可以根据所述首帧视频帧的射影变换矩阵对所述首帧视频帧进行裁剪，得到所述首帧视频帧的稳化视频帧。进而，视频处理设备即可根据首帧视频帧的稳化视频帧和每一帧非首帧视频帧的稳化视频帧，生成稳化视频。
综上可以看出，采用本申请提供的视频稳化方法，通过对每一帧非首帧视频帧的抖动程度的直接矫正，在一定程度上消除了每一帧非首帧视频帧由于抖动所产生的画面畸变。因此，保证了得到的稳化视频相比于待处理视频，在一定程度上消除了由于抖动所产生的画面畸变。
下面结合3个测试示例，对本申请提供的视频稳化方法的效果进行进一步说明。
在第一个测试示例中，测试视频1在拍摄过程中存在旋转运动。如图4所示，为该测试视频1中的一个原始视频帧和该原始视频帧对应的稳化视频帧的对比示意图。其中，图4中的(a)示出了该原始视频帧中若干个点的移动轨迹，图4中的(b)示出了该稳化视频帧中相对应的若干个点的移动轨迹。通过图4的对比，可以看出，采用本申请提供的视频稳化方法对测试视频进行稳化处理后，各个点的抖动幅度明显减小。
获取测试视频1中连续的300帧视频帧的运动参数，和稳化后的测试视频1中对应的 300帧稳化视频帧的稳化运动参数进行统计。其中，参数a的变化曲线如图5(a)所示，参数a对应的稳化值的变化曲线如图5(b)所示。参数b的变化曲线如图6(a)所示，参数b对应的稳化值的变化曲线如图6(b)所示。参数c的变化曲线如图7(a)所示，参数c对应的稳化值的变化曲线如图7(b)所示。参数d的变化曲线如图8(a)所示，参数d对应的稳化值的变化曲线如图8(b)所示。参数e的变化曲线如图9(a)所示，参数e对应的稳化值的变化曲线如图9(b)所示。参数f的变化曲线如图10(a)所示，参数f对应的稳化值的变化曲线如图10(b)所示。经过图5-10的对比可以看出，采用本申请提供的视频稳化方法对测试视频1进行稳化处理后，运动参数中各个参数的抖动幅度明显变小。
在第二个测试示例中，测试视频2存在拍摄过程中在垂直摇晃运动。如图11所示，为该测试视频2中的一个原始视频帧和该原始视频帧对应的稳化视频帧的对比示意图。其中，图11中的(a)示出了该原始视频帧中若干个点的移动轨迹，图11中的(b)示出了该稳化视频帧中相对应的若干个点的移动轨迹。通过图11的对比，可以看出，采用本申请提供的视频稳化方法对测试视频2进行稳化处理后，各个点的上下运动幅度明显减小。
获取测试视频2中连续的300帧视频帧的运动参数，和稳化后的测试视频2中对应的300帧稳化视频帧的稳化运动参数进行统计。其中，参数a的变化曲线如图12(a)所示，参数a对应的稳化值的变化曲线如图12(b)所示。参数b的变化曲线如图13(a)所示，参数b对应的稳化值的变化曲线如图13(b)所示。参数c的变化曲线如图14(a)所示，参数c对应的稳化值的变化曲线如图14(b)所示。参数d的变化曲线如图15(a)所示，参数d对应的稳化值的变化曲线如图15(b)所示。参数e的变化曲线如图16(a)所示，参数e对应的稳化值的变化曲线如图16(b)所示。参数f的变化曲线如图17(a)所示，参数f对应的稳化值的变化曲线如图17(b)所示。经过图12-17的对比可以看出，采用本申请提供的视频稳化方法对测试视频2进行稳化处理后，运动参数中各个参数的抖动幅度明显变小。
在第三个测试示例中，测试视频3在拍摄过程中存在随机抖动的运动。如图18所示，为该测试视频3中的一个原始视频帧和该原始视频帧对应的稳化视频帧的对比示意图。其中，图18中的(a)示出了该原始视频帧中若干个点的移动轨迹，图18中的(b)示出了该稳化视频帧中相对应的若干个点的移动轨迹。通过图18的对比，可以看出，采用本申请提供的视频稳化方法对测试视频3进行稳化处理后，各个点的运动幅度明显减小。
获取测试视频3中连续的300帧视频帧的运动参数，和稳化后的测试视频3中对应的的300帧稳化视频帧的稳化运动参数进行统计。其中，参数a的变化曲线如图19(a)所示，参数a对应的稳化值的变化曲线如图19(b)所示。参数b的变化曲线如图20(a)所示， 参数b对应的稳化值的变化曲线如图20(b)所示。参数c的变化曲线如图21(a)所示，参数c对应的稳化值的变化曲线如图21(b)所示。参数d的变化曲线如图22(a)所示，参数d对应的稳化值的变化曲线如图22(b)所示。参数e的变化曲线如图23(a)所示，参数e对应的稳化值的变化曲线如图23(b)所示。参数f的变化曲线如图24(a)所示，参数f对应的稳化值的变化曲线如24(b)所示。经过图19-24的对比可以看出，采用本申请提供的视频稳化方法对测试视频3进行稳化处理后，运动参数中各个参数的抖动幅度明显变小。
基于上述三个测试示例可知，采用本申请提供的视频稳化方法对测试视频进行稳化处理后，能够有效的缓解视频拍摄过程中由于运动而造成的缩放、旋转、偏移等畸变等。
应理解，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
基于上述实施例所提供的视频稳化方法，本发明实施例进一步给出实现上述方法实施例的装置实施例。
请参见图25，图25是本申请实施例提供的视频处理装置的示意图。包括的各单元用于执行图2对应的实施例中的各步骤。具体请参阅图2对应的实施例中的相关描述。为了便于说明，仅示出了与本实施例相关的部分。参见图9，视频处理装置25包括：
获取单元2501，用于针对待处理视频中的每一非首帧视频帧，获取所述非首帧视频帧的运动参数。
计算单元2502，用于根据所述运动参数计算与所述运动参数对应的几何参数的稳化值，所述几何参数用于描述所述非首帧视频帧的抖动程度；并根据所述运动参数与所述几何参数之间的变换关系，将所述几何参数的稳化值转换为稳化运动参数。
稳化单元2503，用于根据所述稳化运动参数确定所述非首帧视频帧的稳化视频帧，并根据所述待处理视频中的首帧视频帧和每一帧非首帧视频帧各自分别对应的稳化视频帧，生成所述待处理视频对应的稳化视频。
可选的，所述稳化单元2503，还用于在根据所述待处理视频中的首帧视频帧和每一帧非首帧视频帧各自分别对应的稳化视频帧，生成所述待处理视频对应的稳化视频之前，根据预设的裁剪比例以及所述待处理视频的首帧视频帧的高和宽，确定所述首帧视频帧的射影变换矩阵；并根据所述首帧视频帧的射影变换矩阵对所述首帧视频帧进行裁剪，得到所述首帧视频帧的稳化视频帧。
相应的，稳化单元2503根据所述待处理视频中的首帧视频帧和每一帧非首帧视频帧各自分别对应的稳化视频帧，生成所述待处理视频对应的稳化视频，包括：根据所述待处 理视频中的每一帧视频帧各自分别对应的稳化视频帧，生成所述待处理视频对应的稳化视频。
可选的，所述计算单元2502根据所述运动参数计算与所述运动参数对应的几何参数的稳化值，包括：根据所述变换关系，将所述运动参数转换为所述几何参数；对所述几何参数进行平滑滤波处理，得到所述几何参数的稳化值。
可选的，所述几何参数包括在第一方向上的第一缩放比例，在第二方向上的第二缩放比例，偏离所述第一方向的偏第一转角度，偏离所述第二方向的第二偏转角度，在所述第一方向上的第一偏移量，在所述第二方向上的第二偏移量，其中，所述第一方向和所述第二方向相互垂直。
可选的，所述计算单元2502对所述几何参数进行平滑滤波处理，得到所述几何参数的稳化值，包括：对所述第一缩放比例和所述第二缩放比例进行平滑滤波处理，得到所述第一缩放比例的稳化值和所述第二缩放比例的稳化值；对所述第一偏转角度和所述第二偏转角度进行平滑滤波处理，得到的所述第一偏转角度的稳化值和所述第二偏转角度的稳化值；对所述第一偏移量进行平滑滤波处理，得到所述第一偏移量的稳化值；对所述第二偏移量进行平滑滤波处理，得到所述第二偏移量的稳化值。
可选的，所述计算单元2502对所述第一缩放比例和所述第二缩放比例进行平滑滤波处理，得到所述第一缩放比例的稳化值和所述第二缩放比例的稳化值，包括：将所述第一缩放比例和所述第二缩放比例带入预设的第一可逆函数中，计算得到所述第一缩放比例和所述第二缩放比例的均值与第一预设均值的第一差异度；利用第一平滑滤波器对所述第一差异度进行平滑滤波处理，得到所述第一差异度的稳化值；将所述第一缩放比例和所述第二缩放比例带入预设的第二可逆函数中，计算得到所述第一缩放比例和所述第二缩放比例之间的第二差异度；利用第二平滑滤波器对所述第二差异度进行平滑滤波处理，得到所述第二差异度的稳化值；利用所述第一差异度的稳化值和所述第二差异度的稳化值对所述第一可你函数和所述第二可逆函数进行线性求解，得到所述第一缩放比例的稳化值和所述第二缩放比例的稳化值。
可选的，所述计算单元2502对所述第一偏转角度和所述第二偏转角度进行平滑滤波处理，得到的所述第一偏转角度的稳化值和所述第二偏转角度的稳化值，包括：将所述第一偏转角度和所述第二偏转角度带入预设的第三可逆函数中，计算得到所述第一偏转角度和所述第二偏转角度的均值与第三预设均值的第三差异度；利用第三平滑滤波器对所述第三差异度进行平滑滤波处理，得到所述第三差异度的稳化值；将所述第一偏转角度和所述第二偏转角度带入预设的第四可逆函数中，计算得到所述第一偏转角度和所述第二偏转角 度之间的第四差异度；利用第四平滑滤波器对所述第四差异度进行平滑滤波处理，得到所述第四差异度的稳化值；利用所述第三差异度和所述第四差异度对所述第三可逆函数和所述第四可逆函数进行线性求解，得到所述第一偏转角度的稳化值和所述第二偏转角度的稳化值。
可选的，所述计算单元2502对所述第一偏移量进行平滑滤波处理，得到所述第一偏移量的稳化值，包括：将所述第一偏移量带入预设的第五可逆函数，计算所述第一偏移量占据所述非首帧视频帧的画面宽度的第一比例；利用第五平滑滤波器对所述第一比例进行平滑滤波处理，得到一个第一稳化值；利用第六平滑滤波器对所述第一比例进行平滑滤波处理，得到第二稳化值；对所述第一稳化值和所述第二稳化值进行加权计算，得到所述第一偏移量的稳化值；其中，所述第五平滑滤波器的状态转移噪声矩阵参数小于所述第六平滑滤波器的状态噪声矩阵参数。
可选的，所述计算单元2502对所述第一稳化值和所述第二稳化值进行加权计算，得到所述第一偏移量的稳化值，包括：根据所述第五平滑滤波器的状态转移噪声矩阵计算与所述第五平滑滤波器对应的加权值；根据与所述第五平滑滤波器对应的加权值，计算与所述第六平滑滤波器对应的加权值；利用与所述第五平滑滤波器对应的加权值和与所述第六平滑滤波器对应的加权值，对所述第一稳化值和所述第二稳化值进行加权计算，得到所述第一偏移量的稳化值。
可选的，所述计算单元2502根据所述第五平滑滤波器的状态转移噪声矩阵计算与所述第五平滑滤波器对应的加权值，包括：利用公式
Figure PCTCN2020130572-appb-000051
计算与所述第五平滑滤波器对应的加权值；其中，Q 5表示所述第五平滑滤波器的状态转移噪声矩阵，函数
Figure PCTCN2020130572-appb-000052
表示
Figure PCTCN2020130572-appb-000053
的绝对值，dx norm表示所述第一比例，
Figure PCTCN2020130572-appb-000054
表示所述第一稳化值，函数ω()表示差异度计算，0≤w 5≤1。
可选的，所述计算单元2502对所述第二偏移量进行平滑滤波处理，得到所述第二偏移量的稳化值，包括：将所述第二偏移量带入第六可逆函数，计算所述第二偏移量占据所述非首帧视频帧的画面高度的第二比例；利用第七平滑滤波器对所述第二比例进行平滑滤波处理，得到第三稳化值；利用第八平滑滤波器对所述第二比例进行平滑滤波处理，得到第四稳化值；对所述第三稳化值和所述第四稳化值进行加权计算，得到所述第二偏移量的稳化值；其中，所述第七平滑滤波器的状态转移噪声矩阵参数小于所述第八平滑滤波器的状态噪声矩阵参数。
可选的，所述计算单元2502对所述第三稳化值和所述第四稳化值进行加权计算，得到所述第二偏移量的稳化值，包括：利用所述第七平滑滤波器的状态转移噪声矩阵计算与所述第七平滑滤波器对应的加权值；根据与所述第七平滑滤波器对应的加权值，计算与所述第八平滑滤波器对应的加权值；利用与所述第七平滑滤波器对应的加权值和与所述第八平滑滤波器对应的加权值，对所述第三稳化值和所述第四稳化值进行加权计算，得到所述第二偏移量的稳化值。
可选的，所述计算单元2502利用所述第七平滑滤波器的状态转移噪声矩阵计算与所述第七平滑滤波器对应的加权值，包括：利用公式
Figure PCTCN2020130572-appb-000055
计算与所述第七平滑滤波器对应的加权值w 7；其中，Q 7表示所述第七平滑滤波器的状态转移噪声矩阵，函数
Figure PCTCN2020130572-appb-000056
表示
Figure PCTCN2020130572-appb-000057
的绝对值，dy norm表示所述第二比例，
Figure PCTCN2020130572-appb-000058
表示所述第三稳化值，0≤w 7≤1。
可选的，所述稳化单元2503根据所述稳化运动参数确定所述非首帧视频帧的稳化视频帧，包括：根据所述稳化运动参数计算所述非首帧视频帧的射影变换矩阵；利用所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到所述非首帧视频帧的稳化视频帧。
可选的，所述稳化单元2503利用所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到所述非首帧视频帧的稳化视频帧，包括：利用所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到稳化子视频帧；若所述稳化子视频帧的尺寸大小与所述非首帧视频帧的尺寸大小相同，则确定所述稳化子视频帧为所述非首帧视频帧的稳化视频帧。
可选的，所述稳化单元2503利用所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到稳化子视频帧之后，所述方法还包括：若所述稳化子视频帧的尺寸大小与所述非首帧视频帧的尺寸大小不同，则对所述射影变换矩阵进行调整；根据调整后的所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到所述非首帧视频帧的稳化视频帧。
需要说明的是，上述模块之间的信息交互、执行过程等内容，由于与本申请方法实施例基于同一构思，其具体功能及带来的技术效果，具体可参见方法实施例部分，此处不再赘述。
图26是本申请实施例提供的视频处理设备的示意图。如图26所示，该实施例的视频处理设备26包括：处理器260、存储器261以及存储在所述存储器261中并可在所述处理器260上运行的计算机程序262，例如语音识别程序。处理器260执行所述计算机程序262时实现上述各个视频稳化方法实施例中的步骤，例如图2所示的步骤201-204。或者，所述 处理器260执行所述计算机程序262时实现上述各装置实施例中各模块/单元的功能，例如图24所示单元2501-2503的功能。
示例性的，所述计算机程序262可以被分割成一个或多个模块/单元，所述一个或者多个模块/单元被存储在所述存储器261中，并由处理器260执行，以完成本申请。所述一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描述所述计算机程序262在所述视频处理设备26中的执行过程。例如，所述计算机程序262可以被分割成获取单元、计算单元、稳化单元，各单元具体功能请参阅图24对应地实施例中地相关描述，此处不赘述。
所述视频处理设备可包括，但不仅限于，处理器260、存储器261。本领域技术人员可以理解，图26仅仅是视频处理设备26的示例，并不构成对视频处理设备26的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如所述视频处理设备还可以包括输入输出设备、网络接入设备、总线等。
所称处理器260可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
所述存储器261可以是所述视频处理设备26的内部存储单元，例如视频处理设备26的硬盘或内存。所述存储器261也可以是所述视频处理设备26的外部存储设备，例如所述视频处理设备26上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)等。进一步地，所述存储器261还可以既包括所述视频处理设备26的内部存储单元也包括外部存储设备。所述存储器261用于存储所述计算机程序以及所述视频处理设备所需的其他程序和数据。所述存储器261还可以用于暂时地存储已经输出或者将要输出的数据。
本申请实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时可实现上述视频稳化方法。
本申请实施例提供了一种计算机程序产品，当计算机程序产品在视频处理设备上运行时，使得视频处理设备执行时实现可实现上述视频稳化方法。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将所述装置的内部结构划分成不同的功能单元或模块，以完成以上 描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述系统中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
以上所述实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围，均应包含在本申请的保护范围之内。

Claims (19)

  1. 一种视频稳化方法，其特征在于，包括：
    获取待处理视频中的非首帧视频帧的运动参数；
    根据所述运动参数确定与所述运动参数对应的几何参数的稳化值，所述几何参数用于描述所述非首帧视频帧的抖动程度；
    根据所述运动参数与所述几何参数之间的变换关系，将所述几何参数的稳化值转换为稳化运动参数；
    根据所述稳化运动参数确定所述非首帧视频帧的稳化视频帧；
    根据所述稳化视频帧，生成所述待处理视频对应的稳化视频。
  2. 根据权利要求1所述的方法，其特征在于，所述非首帧视频帧包括多帧，且所述多帧非首帧视频帧是连续的；
    相应的，所述根据所述稳化视频帧，生成所述待处理视频对应的稳化视频包括：
    根据每一帧非首帧视频帧各自分别对应的稳化视频帧，生成所述待处理视频对应的稳化视频。
  3. 根据权利要求2所述的方法，其特征在于，所述多帧非首帧视频帧与所述待处理视频的首帧视频帧连续，所述根据每一帧所述非首帧视频帧各自分别对应的稳化视频帧，生成所述待处理视频对应的稳化视频之前，所述方法还包括：
    根据预设的裁剪比例以及所述首帧视频帧的高和宽，确定所述首帧视频帧的射影变换矩阵；
    根据所述首帧视频帧的射影变换矩阵对所述首帧视频帧进行裁剪，得到所述首帧视频帧的稳化视频帧；
    相应的，所述根据每一帧所述非首帧视频帧各自分别对应的稳化视频帧，生成所述待处理视频对应的稳化视频，包括：
    根据所述首帧视频帧的稳化视频帧和每一帧所述非首帧视频帧各自分别对应的稳化视频帧，生成所述待处理视频对应的稳化视频。
  4. 根据权利要求1所述的方法，其特征在于，所述根据所述运动参数确定与所述运动参数对应的几何参数的稳化值，包括：
    根据所述变换关系，将所述运动参数转换为所述几何参数；
    对所述几何参数进行平滑滤波处理，得到所述几何参数的稳化值。
  5. 根据权利要求4所述的方法，其特征在于，所述几何参数包括在第一方向上的第一 缩放比例，在第二方向上的第二缩放比例，偏离所述第一方向的偏第一转角度，偏离所述第二方向的第二偏转角度，在所述第一方向上的第一偏移量，在所述第二方向上的第二偏移量，其中，所述第一方向和所述第二方向相互垂直。
  6. 根据权利要求5所述的方法，其特征在于，所述对所述几何参数进行平滑滤波处理，得到所述几何参数的稳化值，包括：
    对所述第一缩放比例和所述第二缩放比例进行平滑滤波处理，得到所述第一缩放比例的稳化值和所述第二缩放比例的稳化值；
    对所述第一偏转角度和所述第二偏转角度进行平滑滤波处理，得到的所述第一偏转角度的稳化值和所述第二偏转角度的稳化值；
    对所述第一偏移量进行平滑滤波处理，得到所述第一偏移量的稳化值；
    对所述第二偏移量进行平滑滤波处理，得到所述第二偏移量的稳化值。
  7. 根据权利要求6所述的方法，其特征在于，所述对所述第一缩放比例和所述第二缩放比例进行平滑滤波处理，得到所述第一缩放比例的稳化值和所述第二缩放比例的稳化值，包括：
    将所述第一缩放比例和所述第二缩放比例带入预设的第一可逆函数中，计算得到所述第一缩放比例和所述第二缩放比例的均值与第一预设均值的第一差异度；
    利用第一平滑滤波器对所述第一差异度进行平滑滤波处理，得到所述第一差异度的稳化值；
    将所述第一缩放比例和所述第二缩放比例带入预设的第二可逆函数中，计算得到所述第一缩放比例和所述第二缩放比例之间的第二差异度；
    利用第二平滑滤波器对所述第二差异度进行平滑滤波处理，得到所述第二差异度的稳化值；
    利用所述第一差异度的稳化值和所述第二差异度的稳化值对所述第一可你函数和所述第二可逆函数进行线性求解，得到所述第一缩放比例的稳化值和所述第二缩放比例的稳化值。
  8. 根据权利要求6所述的方法，其特征在于，所述对所述第一偏转角度和所述第二偏转角度进行平滑滤波处理，得到的所述第一偏转角度的稳化值和所述第二偏转角度的稳化值，包括：
    将所述第一偏转角度和所述第二偏转角度带入预设的第三可逆函数中，计算得到所述第一偏转角度和所述第二偏转角度的均值与第三预设均值的第三差异度；
    利用第三平滑滤波器对所述第三差异度进行平滑滤波处理，得到所述第三差异度的稳 化值；
    将所述第一偏转角度和所述第二偏转角度带入预设的第四可逆函数中，计算得到所述第一偏转角度和所述第二偏转角度之间的第四差异度；
    利用第四平滑滤波器对所述第四差异度进行平滑滤波处理，得到所述第四差异度的稳化值；
    利用所述第三差异度和所述第四差异度对所述第三可逆函数和所述第四可逆函数进行线性求解，得到所述第一偏转角度的稳化值和所述第二偏转角度的稳化值。
  9. 根据权利要求6所述的方法，其特征在于，所述对所述第一偏移量进行平滑滤波处理，得到所述第一偏移量的稳化值，包括：
    将所述第一偏移量带入预设的第五可逆函数，计算所述第一偏移量占据所述非首帧视频帧的画面宽度的第一比例；
    利用第五平滑滤波器对所述第一比例进行平滑滤波处理，得到一个第一稳化值；
    利用第六平滑滤波器对所述第一比例进行平滑滤波处理，得到第二稳化值；
    对所述第一稳化值和所述第二稳化值进行加权计算，得到所述第一偏移量的稳化值；
    其中，所述第五平滑滤波器的状态转移噪声矩阵参数小于所述第六平滑滤波器的状态噪声矩阵参数。
  10. 根据权利要求9所述的方法，其特征在于，所述对所述第一稳化值和所述第二稳化值进行加权计算，得到所述第一偏移量的稳化值，包括：
    根据所述第五平滑滤波器的状态转移噪声矩阵计算与所述第五平滑滤波器对应的加权值；
    根据与所述第五平滑滤波器对应的加权值，计算与所述第六平滑滤波器对应的加权值；
    利用与所述第五平滑滤波器对应的加权值和与所述第六平滑滤波器对应的加权值，对所述第一稳化值和所述第二稳化值进行加权计算，得到所述第一偏移量的稳化值。
  11. 根据权利要求10所述的方法，其特征在于，所述根据所述第五平滑滤波器的状态转移噪声矩阵计算与所述第五平滑滤波器对应的加权值，包括：
    利用公式
    Figure PCTCN2020130572-appb-100001
    计算与所述第五平滑滤波器对应的加权值；
    其中，Q 5表示所述第五平滑滤波器的状态转移噪声矩阵，函数
    Figure PCTCN2020130572-appb-100002
    表示
    Figure PCTCN2020130572-appb-100003
    的绝对值，dx norm表示所述第一比例，
    Figure PCTCN2020130572-appb-100004
    表示所述第一稳化值，函数ω()表示差异度计算，0≤w 5≤1。
  12. 根据权利要求6所述的方法，其特征在于，所述对所述第二偏移量进行平滑滤波处理，得到所述第二偏移量的稳化值，包括：
    将所述第二偏移量带入第六可逆函数，计算所述第二偏移量占据所述非首帧视频帧的画面高度的第二比例；
    利用第七平滑滤波器对所述第二比例进行平滑滤波处理，得到第三稳化值；
    利用第八平滑滤波器对所述第二比例进行平滑滤波处理，得到第四稳化值；
    对所述第三稳化值和所述第四稳化值进行加权计算，得到所述第二偏移量的稳化值；
    其中，所述第七平滑滤波器的状态转移噪声矩阵参数小于所述第八平滑滤波器的状态噪声矩阵参数。
  13. 根据权利要求12所述的方法，其特征在于，所述对所述第三稳化值和所述第四稳化值进行加权计算，得到所述第二偏移量的稳化值，包括：
    利用所述第七平滑滤波器的状态转移噪声矩阵计算与所述第七平滑滤波器对应的加权值；
    根据与所述第七平滑滤波器对应的加权值，计算与所述第八平滑滤波器对应的加权值；
    利用与所述第七平滑滤波器对应的加权值和与所述第八平滑滤波器对应的加权值，对所述第三稳化值和所述第四稳化值进行加权计算，得到所述第二偏移量的稳化值。
  14. 根据权利要求13所述的方法，其特征在于，所述利用所述第七平滑滤波器的状态转移噪声矩阵计算与所述第七平滑滤波器对应的加权值，包括：
    利用公式
    Figure PCTCN2020130572-appb-100005
    计算与所述第七平滑滤波器对应的加权值w 7；
    其中，Q 7表示所述第七平滑滤波器的状态转移噪声矩阵，函数
    Figure PCTCN2020130572-appb-100006
    表示
    Figure PCTCN2020130572-appb-100007
    的绝对值，dy norm表示所述第二比例，
    Figure PCTCN2020130572-appb-100008
    表示所述第三稳化值，0≤w 7≤1。
  15. 根据权利要求1所述的方法，其特征在于，所述根据所述稳化运动参数确定所述非首帧视频帧的稳化视频帧，包括：
    根据所述稳化运动参数计算所述非首帧视频帧的射影变换矩阵；
    利用所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到所述非首帧视频帧的稳化视频帧。
  16. 根据权利要求1-15任一项所述的方法，其特征在于，所述利用所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到所述非首帧视频帧的稳化视频帧，包括：
    利用所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到稳化子视频帧；
    若所述稳化子视频帧的尺寸大小与所述非首帧视频帧的尺寸大小相同，则确定所述稳化子视频帧为所述非首帧视频帧的稳化视频帧。
  17. 根据权利要求16所述的方法，其特征在于，所述利用所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到稳化子视频帧之后，所述方法还包括：
    若所述稳化子视频帧的尺寸大小与所述非首帧视频帧的尺寸大小不同，则对所述射影变换矩阵进行调整；
    根据调整后的所述射影变换矩阵对所述非首帧视频帧进行裁剪，得到所述非首帧视频帧的稳化视频帧。
  18. 一种视频处理设备，其特征在于，处理器、存储器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如权利要求1至17任一项所述的方法。
  19. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至17任一项所述的方法。
PCT/CN2020/130572 2020-03-19 2020-11-20 一种视频稳化方法及装置 WO2021184812A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010195539.2A CN113497861B (zh) 2020-03-19 2020-03-19 一种视频稳化方法及装置
CN202010195539.2 2020-03-19

Publications (1)

Publication Number Publication Date
WO2021184812A1 true WO2021184812A1 (zh) 2021-09-23

Family

ID=77769691

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/130572 WO2021184812A1 (zh) 2020-03-19 2020-11-20 一种视频稳化方法及装置

Country Status (2)

Country Link
CN (1) CN113497861B (zh)
WO (1) WO2021184812A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117689569A (zh) * 2022-08-16 2024-03-12 腾讯科技（深圳）有限公司 视频帧的处理方法、装置、计算机设备和存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101951463A (zh) * 2010-05-19 2011-01-19 上海稳像信息技术有限公司 基于简易快速全局运动参数估计的实时视频稳像方法
CN103841297A (zh) * 2012-11-23 2014-06-04 中国航天科工集团第三研究院第八三五七研究所 一种适用于合成运动摄像载体的电子稳像方法
US20140226070A1 (en) * 2013-02-08 2014-08-14 Ati Technologies Ulc Method and apparatus for reconstructing motion compensated video frames

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI469062B (zh) * 2011-11-11 2015-01-11 Ind Tech Res Inst 影像穩定方法及影像穩定裝置
CN102724387B (zh) * 2012-05-26 2016-08-03 安科智慧城市技术（中国）有限公司 一种电子稳像的方法及装置
US9756249B1 (en) * 2016-04-27 2017-09-05 Gopro, Inc. Electronic image stabilization frequency estimator
CN106550174B (zh) * 2016-10-28 2019-04-09 大连理工大学 一种基于单应性矩阵的实时视频稳像方法
CN110796010B (zh) * 2019-09-29 2023-06-06 湖北工业大学 一种结合光流法和卡尔曼滤波的视频稳像方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101951463A (zh) * 2010-05-19 2011-01-19 上海稳像信息技术有限公司 基于简易快速全局运动参数估计的实时视频稳像方法
CN103841297A (zh) * 2012-11-23 2014-06-04 中国航天科工集团第三研究院第八三五七研究所 一种适用于合成运动摄像载体的电子稳像方法
US20140226070A1 (en) * 2013-02-08 2014-08-14 Ati Technologies Ulc Method and apparatus for reconstructing motion compensated video frames

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
FANG MING; LI HAOYUE; SI SHUZHE: "A Video Stabilization Algorithm Based on Affine SIFT", 2018 INTERNATIONAL CONFERENCE ON COMPUTING, MATHEMATICS AND ENGINEERING TECHNOLOGIES (ICOMET), 3 March 2018 (2018-03-03), pages 1 - 4, XP033335275, DOI: 10.1109/ICOMET.2018.8346332 *
XU JIE , YANG SHUO , WANG MING-HUI: "Digital Video Stabilization based on Jittery Frame Checking and Low-Pass Filtering", SICHUAN DAXUE XUEBA, ZIRAN KEXUE - ACTA SCIETIARUM NATURALIUMUNIVERSITATIS SZECHUANENSIS, vol. 50, no. 6, 28 November 2013 (2013-11-28), CN, pages 1235 - 1240, XP055851360, ISSN: 0490-6756, DOI: 10.3969/j.issn.0490-6756.2013.06.015 *

Also Published As

Publication number Publication date
CN113497861B (zh) 2022-06-24
CN113497861A (zh) 2021-10-12

Similar Documents

Publication Publication Date Title
US10887519B2 (en) Method, system and apparatus for stabilising frames of a captured video sequence
US10915998B2 (en) Image processing method and device
WO2018223381A1 (zh) 一种视频防抖方法及移动设备
US8447140B1 (en) Method and apparatus for estimating rotation, focal lengths and radial distortion in panoramic image stitching
US9516228B2 (en) Photographing apparatus, motion estimating apparatus, image compensating method, motion estimating method, and computer-readable recording medium
US11012608B2 (en) Processing method and mobile device
WO2015030221A1 (en) Image processing apparatus, image processing method, and imaging system
WO2013121897A1 (ja) 情報処理装置および方法、画像処理装置および方法、並びにプログラム
CN104601892A (zh) 一种终端、图像拍摄方法及装置
US9615040B2 (en) Determining a maximum inscribed size of a rectangle
CN113556464B (zh) 拍摄方法、装置及电子设备
WO2021189804A1 (zh) 图像矫正方法、装置和电子系统
JP2010041419A (ja) 画像処理装置、画像処理プログラム、画像処理方法、および電子機器
US8340469B2 (en) Information processing apparatus, information processing method, program, and image processing apparatus
US20200160560A1 (en) Method, system and apparatus for stabilising frames of a captured video sequence
US20120287311A1 (en) Digital image processing device and processing method thereof
JP2015046044A (ja) 画像処理装置、画像処理方法、プログラムおよび撮像システム
WO2018058476A1 (zh) 一种图像校正方法及装置
WO2021184812A1 (zh) 一种视频稳化方法及装置
WO2023005355A1 (zh) 图像防抖方法与电子设备
JP2016053978A (ja) 画像処理装置
WO2023221969A1 (zh) 3d图片拍摄方法和3d拍摄系统
WO2021109863A1 (zh) 照片处理方法及照片处理装置
WO2023142732A1 (zh) 图像处理方法、装置和电子设备
JP2015154334A (ja) 撮像装置、その制御方法、および制御プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20925915

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20925915

Country of ref document: EP

Kind code of ref document: A1