video jitter elimination method and device, terminal equipment and storage medium
Technical Field
The present invention relates to the field of video processing technologies, and in particular, to a method and an apparatus for eliminating video jitter, a terminal device, and a storage medium.
Background
Video jitter refers to the overall displacement between video frames, thereby affecting a series of subsequent operations such as video impression and compression coding.
Technologies for detecting video jitter are currently roughly: an optical flow method, a block matching method, a feature point matching method, and a gray projection method. Fast corner detection and LK sparse optical flow are adopted in the actual operation of the optical flow method. Block matching is a common method in image denoising and motion estimation. By matching the query block with neighboring image blocks, the K blocks closest to the query block are found from these neighboring blocks. The feature point matching method and the optical flow method are relatively dependent on the search of feature points. The gray projection method is a method which is used more, has relatively small calculated amount and has good actual effect.
The optical flow method depends on the detection of characteristic points, and the robustness is poor; the block matching method has higher requirements on image stability; the feature point matching method also depends on the search of feature points, and the calculated amount is huge; the gray projection method utilizes the change of the overall gray distribution rule of the image to determine the movement between frames, but has the defects that the global motion calculation is influenced by foreground motion and the effect on the rotary motion is poor.
Disclosure of Invention
In view of the above-mentioned deficiencies of the prior art, the present invention provides a video jitter elimination method, an elimination apparatus, a terminal device and a storage medium.
the technical scheme adopted by the invention is as follows:
In a first aspect, the present invention provides a video jitter elimination method, including:
Dividing the video frame into a plurality of sub-region blocks, and screening the sub-region blocks to obtain a screened video frame;
And performing motion compensation on the screened video frame by adopting motion estimation to obtain a compensated video frame.
in a first possible implementation manner of the first aspect, the video frame is divided into a plurality of sub-region blocks, and the sub-region blocks are screened to obtain a screened video frame; the method specifically comprises the following steps:
And screening the subarea blocks by adopting gray gradient, deleting the subarea blocks with jitter to obtain blank subarea blocks, reserving the subarea blocks without jitter, and forming a screening video frame by the blank subarea blocks and the subarea blocks without jitter.
In a second possible implementation manner of the first aspect, the screening the sub-region blocks by using a gray scale gradient specifically includes:
and calculating the gray gradient of each sub-region block, and comparing the gray gradient of each sub-region block with a gradient threshold, wherein the sub-region block with the gray gradient smaller than the gray threshold is a sub-region block with jitter, and the sub-region block with the gray gradient larger than or equal to the gray threshold is a sub-region block without jitter.
Further, the gray threshold is an average value of gray gradients of the sub-region blocks.
in a third possible implementation manner of the first aspect, the screening the sub-region blocks by using a gray scale gradient specifically includes:
Calculating the gray scale gradient of each sub-region block, and sequencing the gray scale gradients according to the numerical value;
Calculating the difference between every two adjacent gray gradients according to the sorting sequence;
Calculating the average value of all the difference values, and deleting the sub-region module corresponding to the difference value larger than the average value;
The sub-region module corresponding to the difference value is a sub-region module corresponding to a larger value of the two gray gradients used for calculating the difference.
In a fourth possible implementation manner of the first aspect, the motion compensation is performed on the screened video frames by using motion estimation to obtain compensated video frames; the method specifically comprises the following steps: and obtaining the motion information of the video frame by adopting motion estimation, and performing motion compensation on the screened video frame.
in a second aspect, an embodiment of the present invention provides a video jitter removing apparatus, including:
the screening module is used for dividing the video frame into a plurality of sub-area blocks and screening the sub-area blocks to obtain a screened video frame;
And the compensation module is used for performing motion compensation on the screened video frames by adopting motion estimation to obtain compensated video frames.
in a third aspect, an embodiment of the present invention provides a video jitter removing apparatus, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the video jitter removing method when executing the computer program.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored, where the computer program is executed by a processor to implement a video jitter elimination method as described above.
The invention has the beneficial effects that:
The gray scale gradient of each sub-region block is calculated, and the sub-region blocks with the gray scale gradient smaller than the gradient threshold value are deleted. Video jitter is identified according to the gray gradient, and the algorithm is simple; motion compensation is carried out through motion estimation, so that the quality of finally obtained video frames is ensured; the gradient threshold value is a variable value, and the updating of the gradient threshold value according to each frame of video is realized, so that the screening of the sub-region blocks in each video frame is favorably ensured to be carried out according to the characteristics of the video frame, the video jitter elimination method is ensured to adapt according to the characteristics of the video frame, the video jitter elimination effect is favorably further improved, the integrity of the video is ensured, and the quality of each frame of video is ensured.
Drawings
FIG. 1 is a flow chart of a video jitter removal method according to the present invention;
fig. 2 is a block diagram of a video jitter removing apparatus according to the present invention.
Detailed Description
The invention provides a video jitter elimination method, which realizes the identification and elimination of jitter blur in a video frame by screening a sub-region module according to gray gradient; after the jitter blur is deleted, the blank area is compensated according to the motion estimation, so that the integrity of the video frame is ensured, and the video jitter is eliminated.
The foregoing is the core idea of the present application, and in order to make those skilled in the art better understand the scheme of the present application, the present application will be further described in detail with reference to the accompanying drawings. It should be understood that the specific features in the embodiments and examples of the present application are detailed description of the technical solutions of the present application, and are not limited to the technical solutions of the present application, and the technical features in the embodiments and examples of the present application may be combined with each other without conflict.
Example one
Fig. 1 is a flow chart of a video jitter elimination method according to the present invention.
Referring to fig. 1, a video judder removal method includes the steps of:
step 101, dividing a video frame into a plurality of sub-area blocks, and screening the sub-area blocks to obtain a screened video frame;
And 102, performing motion compensation on the screened video frame by adopting motion estimation to obtain a compensated video frame.
therefore, in the invention, the video jitter is identified according to the gray gradient, and the algorithm is simple; and the quality of the finally obtained video frame is ensured by performing motion compensation through motion estimation. In addition, the invention uses the video frames to carry out the jitter elimination object, thereby ensuring the integrity of the video and the quality of each frame of video.
In step 101, dividing a video frame into a plurality of sub-area blocks, and screening the sub-area blocks to obtain a screened video frame; the method specifically comprises the following steps:
and screening the subarea blocks by adopting gray gradient, deleting the subarea blocks with jitter to obtain blank subarea blocks, reserving the subarea blocks without jitter, and forming a screening video frame by the blank subarea blocks and the subarea blocks without jitter.
The screening of the sub-region blocks by using the gray scale gradient specifically comprises:
And calculating the gray gradient of each sub-region block, and comparing the gray gradient of each sub-region block with a gradient threshold, wherein the sub-region block with the gray gradient smaller than the gray threshold is a sub-region block with jitter, and the sub-region block with the gray gradient larger than or equal to the gray threshold is a sub-region block without jitter.
The gray threshold is the mean value of the gray gradient of each sub-region block.
In this embodiment, the specific method for selecting the sub-region blocks according to the gray scale gradient includes: and calculating the gray gradient of each sub-region block, and deleting the sub-region blocks with the gray gradient smaller than a gradient threshold value. Specifically, in this embodiment, the gradient threshold is an average value of the block gray scale gradients of the sub-regions. In the embodiment, the gradient threshold is a variable value, so that the gradient threshold is updated according to each frame of video, thereby being beneficial to ensuring that screening of sub-region blocks in each video frame is carried out according to the characteristics of the video frame, ensuring the characteristic that the video jitter elimination method adapts according to the video frame, and being beneficial to further improving the video jitter elimination effect.
example two
In this embodiment, another implementation manner is adopted for the sub-region block screening on the basis of the first embodiment, and specifically, the implementation manner is as follows: screening the subarea blocks by adopting gray gradient, which specifically comprises the following steps:
Calculating the gray scale gradient of each sub-region block, and sequencing the gray scale gradients according to the numerical value;
calculating the difference between every two adjacent gray gradients according to the sorting sequence;
calculating the average value of all the difference values, and deleting the sub-region module corresponding to the difference value larger than the average value;
the sub-region module corresponding to the difference value is a sub-region module corresponding to a larger value of the two gray gradients used for calculating the difference.
In this embodiment, it is assumed that a video frame is divided into N sub-region modules, and the gray scale gradients corresponding to the sub-region modules are: a1, a2, a3 … …, an; a1, a2, a3 … … and an are arranged according to numerical value size sequence, a2-a1= c1, a3-a2= c2 … …, an-a (n-1) = c (n-1); the sub-region block corresponding to c1 is the sub-region block with the gray scale gradient of a2, the sub-region block corresponding to c2 is the sub-region block with the gray scale gradient of a3, and the sub-region block corresponding to c (n-1) is the sub-region block with the gray scale gradient of an.
Assuming that the average value of c1, c2 … … and c (n-1) is c0, wherein c2 is greater than c0, the sub-region block with the gray gradient of a3 is eliminated in the video frame, and the video frame after the sub-region block is eliminated is subjected to motion compensation through motion estimation.
In this embodiment, the screening of the sub-region blocks is performed according to the gray gradient difference, so that the screening accuracy of the video jitter is further improved.
In step 102, performing motion compensation on the screened video frame by using motion estimation to obtain a compensated video frame; the method specifically comprises the following steps: and obtaining the motion information of the video frame by adopting motion estimation, and performing motion compensation on the screened video frame.
according to the video jitter elimination method, the sub-region module is screened according to the gray gradient, so that the identification and elimination of jitter blur in the video frame are realized; after the jitter blur is deleted, the blank area is compensated according to the motion estimation, so that the integrity of the video frame is ensured, and the video jitter is eliminated.
EXAMPLE III
Based on the same inventive concept as the video jitter removing method in the foregoing embodiment, the present invention also provides a video jitter removing apparatus.
referring to fig. 2, a block diagram of a video jitter removing apparatus according to the present invention is shown.
As shown in fig. 2, a video judder removing apparatus includes:
the screening module 201 is configured to divide a video frame into a plurality of sub-region blocks, and screen the sub-region blocks to obtain a screened video frame;
and the compensation module 202 is configured to perform motion compensation on the screened video frame by using motion estimation to obtain a compensated video frame.
Various modifications and embodiments of a video jitter removal method in the first embodiment are also applicable to a video jitter removal method in the present embodiment, and a video jitter removal method in the present embodiment is clearly known to those skilled in the art from the foregoing detailed description of a video jitter removal method, so that details are not described herein again for the sake of brevity of the description.
Example four
A video judder removal device comprising a memory, a processor and a computer program stored in said memory and executable on said processor, characterized in that said processor implements a video judder removal method as described when executing said computer program.
the Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, a discrete hardware component, or the like. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, said processor being the control center for said one video jitter removal method, various interfaces and lines connecting the various parts of the overall said one video jitter removal method.
The memory can be used for storing the computer program and/or the module, and the processor can realize various functions of the video structuring storage device based on edge computing by operating or executing the computer program and/or the module stored in the memory and calling the data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, a text conversion function, etc.), and the like; the storage data area may store data (such as audio data, text message data, etc.) created according to the use of the cellular phone, etc. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
EXAMPLE five
A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out a video judder removal method as described.
wherein, the module integrated by the video jitter elimination method can be stored in a computer readable storage medium if the module is realized in the form of a software functional unit and sold or used as a stand-alone product. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
The above examples are typical examples of the present invention, but the embodiments of the present invention are not limited to the above examples. Other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principles of the invention are intended to be included within the scope of the invention.
the present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
finally, it should be noted that: the above embodiments are only intended to illustrate the technical solution of the present invention and not to limit the same, and a person of ordinary skill in the art can make modifications or equivalents to the specific embodiments of the present invention with reference to the above embodiments, and such modifications or equivalents without departing from the spirit and scope of the present invention are within the scope of the claims of the present invention as set forth in the claims.