CN108282653B - Motion compensation de-interlacing method and system based on motion estimation of bipolar field - Google Patents
Motion compensation de-interlacing method and system based on motion estimation of bipolar field Download PDFInfo
- Publication number
- CN108282653B CN108282653B CN201810117131.6A CN201810117131A CN108282653B CN 108282653 B CN108282653 B CN 108282653B CN 201810117131 A CN201810117131 A CN 201810117131A CN 108282653 B CN108282653 B CN 108282653B
- Authority
- CN
- China
- Prior art keywords
- field
- motion
- vector
- image
- candidate vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
- H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
- H04N19/16—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter for a given display mode, e.g. for interlaced or progressive display mode
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/513—Processing of motion vectors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Television Systems (AREA)
Abstract
The invention discloses a motion compensation de-interlacing method and a system based on motion estimation of a bipolar field, wherein the method comprises the following steps: performing 3DRS motion estimation between input homopolar field images cf and p2, and performing first search to obtain a converged homopolar field motion vector field of the current field cf; performing second search on 3DRS motion estimation between the images p2' and p1 to make the images converge on a real motion vector, and obtaining a heteropolar field motion vector field of the current interpolation field p 1; the invention not only can overcome the problems of high calculation complexity and low convergence speed of the existing motion estimation, but also can solve the problems of interlaced flicker of the existing repeated frame scene of the same polarity field and interlaced flicker, and inconsistency of the motion vectors of the different polarity fields in the time domain, which causes interlaced errors, sawteeth and feathering.
Description
Technical Field
The invention relates to the field of video image processing, in particular to a motion compensation de-interlacing method and a motion compensation de-interlacing system based on 3DRS (3-dimensional Recursive Search) motion estimation of a bipolar field.
Background
Conventional Television systems based on Interlaced Scanning (Interlaced Scanning) video signals have appeared in the 30's of the 20 th century and have been used for decades now, and most of the video systems now record or convert video mainly in the PAL (Phase alternation Line), NTSC (National Television Standards Committee) format. The interlaced scanning technique is to divide each frame image into a Top Field (Top Field) and a Bottom Field (Bottom Field) to display Field images in an interlaced manner, wherein the interlaced scanning frequency is half of the progressive scanning frequency, and the frequency spectrum of the video signal and the channel bandwidth for transmitting the video signal are also half of the progressive scanning frequency. Due to the persistence of vision effect, human eyes will see smooth motion rather than flickering field images, which effectively increases the utilization rate of channels under the condition that the subjective thinking is that the image quality is not reduced much. However, interlaced scanning has some disadvantages: the defects of moving object blur, boundary flicker, feathering, picture jitter, sawtooth on bevel edge and the like.
With the development and maturity of digital television and high definition television technologies and the gradual improvement of video quality requirements of people, a progressive scanning mode has become a preferred scheme of a digital television scanning mode, and a current novel flat panel display terminal is also a display device supporting progressive scanning. However, a large number of video files recorded in an interlaced manner have been left before, and thus, it is necessary to convert them into progressive video files, and the deinterlace technique is a video format conversion technique for converting interlaced signals into progressive signals.
Currently, de-interlacing techniques are mainly divided into three main categories: de-interlacing based on spatial domain 2D interpolation technology, de-interlacing based on temporal domain 2D interpolation technology, and de-interlacing based on temporal domain 3D interpolation technology. The spatial domain 2D interpolation is to restore the pixel points to be processed through the in-field information; the time domain 2D interpolation is to restore an image by utilizing the correlation among different fields; the time-space domain 3D interpolation is the interpolation by comprehensively utilizing information between fields and information in fields, and the algorithm mainly comprises the following steps: the method comprises the following steps of content self-adaptation, motion compensation and other algorithms, wherein the motion compensation algorithm can well keep the time domain and space domain details of a moving object and is the most advanced de-interlacing algorithm at present.
On the one hand, Motion Estimation (Motion Estimation) is used in many video processing techniques to find the correct Motion Vector (Motion Vector), such as Motion compensation in MPEG-2 video interframe predictive coding and Motion compensated de-interlacing in de-interlacing. Motion estimation is to find the best reference Block matching the currently processed Block (Block Match). At present, fast algorithms such as a Full Search (FS), a three-step method (TSS), a four-step method (FSs), a diamond method (DS), a multilayer pyramid 3DRS and the like are most adopted. However, these algorithms have defects, and the full search computation complexity is too high; three-step (TSS), four-step (FSS), Diamond (DS) methods have slow convergence rates; and the multilayer pyramid 3DRS has high calculation complexity, and can reach a certain convergence rate by multilayer repeated search.
On the other hand, the motion compensated de-interlacing method in the market at present calculates the motion vector field of the current field by motion estimation in the same polarity field (consecutive adjacent top field-top field or bottom field-bottom field), and then maps the pixels in the reference field according to the motion vector of the current field as the de-interlaced motion compensated interpolation pixels of the current field, but this method has a fatal problem: usually, the motion between two adjacent fields (fields with different polarities) is inconsistent, and it is impossible to simply perform motion compensation by using the field with the same polarity, which easily causes compensation errors, such as: the fields with the same polarity are repeated frames (repeat frames), and there is motion between fields with different polarities, and at this time, the motion vector (i.e. 0Mv vector) estimated by the motion of the fields with the same polarity is used for carrying out compensation interpolation de-interlacing on fields with different polarities, which can cause image defects such as flicker and feathering of de-interlaced images, and thus human eyes can feel uncomfortable and visual fatigue when watching such images.
Disclosure of Invention
In order to overcome the defects of the prior art, the present invention provides a motion compensation de-interlacing method and system based on motion estimation of a bipolar field, so as to solve the problem of de-interlacing flicker of repeated frame scenes of a field with the same polarity in the prior art, and the problems of de-interlacing errors, jaggies and feathering caused by the inconsistency of motion vectors of the field with the same polarity and motion vectors of a field with different polarity in a time domain.
Another objective of the present invention is to provide a motion compensation de-interlacing method and system based on motion estimation of bipolar field to overcome the problems of high computational complexity and slow convergence rate of the existing motion estimation techniques.
To achieve the above and other objects, the present invention provides a motion compensated de-interlacing method based on motion estimation of bipolar fields, comprising the steps of:
firstly, performing 3DRS motion estimation between input homopolar field images cf and p2, and performing first search to obtain a converged homopolar field motion vector field of a current field cf, wherein cf represents a current field image, and p2 represents a reference field image for motion estimation;
secondly, performing second search on 3DRS motion estimation between the images p2 'and p1, further performing accurate motion vector search to make the images converge on a real motion vector, and obtaining a heteropolar field motion vector field of the current interpolation field p1, wherein p1 represents a currently input field image, and p2' is a reference frame image of the heteropolar field motion estimation;
and step three, projecting the motion vector of the field with different polarity onto a reference frame p2', finding a reference block pixel which is best matched with the current block to perform motion compensation de-interlacing on the current compensation interpolation field p1, and outputting a current de-interlacing result.
Preferably, the step one further comprises the steps of:
segmenting an input field image cf according to an image block unit, wherein cf is a current field image;
taking an input p1 field block motion vector as a time domain candidate vector of a current field, taking a segmented image block as a basic unit and a p2 field image signal as a reference field, and selecting and optimizing multi-class candidate vectors based on a space domain, a time domain, 0Mv and a random motion candidate vector to carry out homopolar field 3DRS motion estimation, wherein p1 represents a previous field image, and p2 represents a previous field image;
the same-polarity block motion vector field of the current field cf is output.
Preferably, the homopolar field 3DRS motion estimation further comprises the steps of:
step S2.1, according to the segmented current image block cfblkSelecting a plurality of types of candidate vectors of a space domain candidate vector, a time domain candidate vector, a 0Mv candidate vector and a random motion candidate vector;
step S2.2, calculating SAD value of each candidate vector;
s2.3, calculating the punishment quantity of each candidate vector according to the SAD value of each candidate vector;
s2.4, selecting an optimal candidate vector according to the SAD value of each candidate vector and the penalty of each candidate vector;
s2.5, calculating a self-adaptive search step size factor according to the selected optimal candidate vector;
and S2.6, selecting an optimal updating candidate vector according to the updating candidate base vector and the self-adaptive searching step-size factor, and outputting the optimal updating candidate vector to a block motion vector cache of the current field to be used as a space domain candidate vector, a neighborhood candidate vector and a time domain candidate vector of the next image block.
Preferably, in step S2.2, the image is selected according to the current image block cfblkAnd the motion magnitudes of the spatial domain candidate vector, the temporal domain candidate vector, the 0Mv candidate vector and the random motion candidate vector are respectively projected onto a homopolar field p2 to obtain a corresponding image block p2blkRespectively calculating its corresponding image block and its preceding image block cfblkThe symbol is denoted as SAD.
Preferably, step 2.3 further comprises:
respectively calculating respective penalty factors according to the types of the candidate vectors;
calculating the distance between each candidate vector and the neighborhood motion vector of the current block;
and respectively calculating the penalty amount of each candidate vector according to each penalty factor and the distance between each candidate vector and the neighborhood motion vector of the current block.
Preferably, step S2.4 further comprises:
calculating to obtain the corresponding graph of each candidate vector according to the SAD of each candidate vector and the punishment of each candidate vectorBlock matching value SADP；
According to the constrained SAD of each candidate vectorPAnd selecting the optimal candidate vector.
Preferably, in step S2.5, the calculation formula of the updated search step factor is as follows:
β＝max[1,min(βmax,λ·SADmin)]
wherein, λ is a preset coefficient of the step factor, βmaxFor a preset maximum threshold value of the step-size factor, beta denotes the adaptive search step-size factor, SADminIs the optimal candidate vector.
Preferably, step 2.6 further comprises:
calculating an updated candidate vector U according to the updated candidate base vector and the self-adaptive search step size factori；
Respectively calculating an update candidate vector U according to the step S2.2iThe corresponding SAD value;
and selecting the optimal updating candidate vector according to the SAD value of the updating candidate vector.
Preferably, the second step further comprises the steps of:
splitting an input field image p1 in units of image blocks, wherein p1 represents a previous field image;
performing field interpolation on the field-level image blocks to obtain frame-level image blocks;
taking image blocks after intra-field interpolation as a basic unit, p2' as a reference field, and taking an input p1 field block motion vector as a time domain candidate vector of a current field, and performing heteropolarity field 3DRS motion estimation; wherein p2' is a reference frame image for heteropolar field motion estimation;
the block motion vector field of the opposite polarity field p1 is output.
To achieve the above object, the present invention further provides a motion compensated de-interlacing system based on motion estimation of bipolar fields, comprising:
a homopolar field motion estimation unit, configured to perform a first search on 3DRS motion estimation between input homopolar field images cf and p2 to obtain a homopolar field motion vector field of a converged current field cf, where cf represents a current field image and p2 represents a previous field image;
the heteropolar field motion estimation unit is used for carrying out second search on 3DRS motion estimation between the images p2 'and p1, further accurately searching a motion vector to make the motion vector converge on a real motion vector, and obtaining a heteropolar field motion vector field of the current interpolation field p1, wherein p1 represents a previous field image, and p2' represents a previous frame image obtained after motion compensation;
and the motion compensation de-interlacing unit is used for projecting the motion vectors of the fields with different polarities onto the reference frame p2', finding out the reference block pixel which is best matched with the current block to perform motion compensation de-interlacing on the current compensation interpolation field p1, and outputting a current de-interlacing result.
Compared with the prior art, the motion compensation de-interlacing method and system based on the motion estimation of the bipolar field perform first motion vector search through the motion estimation of the 3DRS of the same polarity field, so that the motion vector field is converged quickly, then perform motion compensation interpolation de-interlacing processing according to the vector field searched by the 3DRS of the different polarity field, further refine the motion vector of the current field motion compensation, so that the motion vector is converged to a real motion vector, and finally perform motion compensation interpolation de-interlacing processing according to the vector field searched by the 3DRS of the different polarity field, so that the de-interlacing flicker problem phenomenon of a repeated frame scene of the same polarity field is solved, and the de-interlacing problem phenomena of errors, sawteeth and feathering caused by the fact that the motion vector of the same polarity field and the motion vector of the different polarity field are different in the time domain are solved.
Drawings
FIG. 1 is a flowchart illustrating the steps of a motion compensated de-interlacing method based on motion estimation of bipolar fields according to the present invention;
FIG. 2 is a detailed flowchart of step S1 according to an embodiment of the present invention;
FIG. 3 is a block diagram of an input field image according to an embodiment of the present invention;
fig. 4 is a flowchart of the same polarity field 3DRS motion estimation of step S102 in an embodiment of the present invention;
FIG. 5 is a detailed diagram illustrating the selection of multi-class candidate vectors according to an embodiment of the invention;
FIG. 6 is a diagram illustrating neighborhood motion vectors in an embodiment of the present invention;
FIG. 7 is a diagram illustrating updating candidate base vectors according to an embodiment of the present invention;
FIG. 8 is a detailed flowchart of step S2 according to an embodiment of the present invention;
FIG. 9 is a diagram illustrating motion compensation according to an embodiment of the present invention;
FIG. 10 is a diagram of the system architecture of a motion compensated de-interlacing system based on motion estimation of bipolar fields according to the present invention;
FIG. 11 is a diagram illustrating operation of the motion compensated de-interlacing based on motion estimation of bipolar fields in FIG. 10;
FIG. 12 is a diagram illustrating a homopolar field motion estimation unit according to an embodiment of the present invention;
fig. 13 is a diagram of an heteropolar field motion estimation unit according to an embodiment of the present invention.
Detailed Description
Other advantages and capabilities of the present invention will be readily apparent to those skilled in the art from the present disclosure by describing the embodiments of the present invention with specific embodiments thereof in conjunction with the accompanying drawings. The invention is capable of other and different embodiments and its several details are capable of modification in various other respects, all without departing from the spirit and scope of the present invention.
Because the convergence speed and accuracy of a motion vector field directly influence the image effect of motion compensation de-interlacing, according to the correlation of a time domain and a space domain of the motion vector field, the invention provides the selection and optimization of a plurality of types of motion candidate vectors based on the space domain, the time domain, 0Mv and random motion candidate vectors, the convergence speed of the 3DRS is greatly accelerated, which is one of key points of the invention, and the problem of the difficult solution of the prior method for searching the motion vector based on the 3DRS, wherein the convergence speed is slow, is solved. Fig. 1 is a flowchart illustrating the steps of a motion compensated de-interlacing method based on motion estimation of bipolar fields according to the present invention. As shown in fig. 1, the present invention relates to a motion compensation de-interlacing method based on motion estimation of bipolar field, which comprises the following steps:
step S1, perform a first search for motion estimation of the 3DRS between the input homopolar field images cf and p2 to obtain a motion vector field (homopolar field motion vector field) of the current field cf that converges.
Specifically, as shown in fig. 2, step S1 further includes:
in step S101, the input field image cf is divided into image blocks (H'blk×W′blkA block of pixels of size) unit slice, where cf (current field) is the currently input field image.
The image resolutions employed by particular embodiments of the present invention are illustrated by way of example (and certainly not limited to other resolution sizes), with an input frame image pixel resolution size of 1920 × 1080, an input field image pixel resolution size of 540 × 1920, and an image block resolution size of W'blk×H′blk(wherein H'blk＝4,W′blk8), the image data is calculated in the luminance Y component of the YCbCr color space. Specifically, a block diagram of an input field picture is shown in fig. 3, cf is a currently input field picture, p2 is a reference field picture for motion estimation, the field picture resolution is set to 1920 × 540 pixels, the image is divided into units of pixel blocks (pixel blocks of 8 × 4 size), and there are M × N (M is 1920/8 is 240, N is 1080/4 is 135) image blocks (blocks) in total; cf (j, i) represents the ith row and jth column of pixels of the cf field image; cfblkThe method includes the steps that an nth row and an mth column block in a current field cf image are represented (the value range of N is {0, 1.., N-1}, and the value range of M is {0, 1.., M-1}), cf are obtainedblk(l, k) represents the current block cfblkThe kth row and the l column of pixels.
Step S102, taking an input p1 field block motion vector (here, an iterative algorithm, where the motion vector of the cf field is currently being searched, which indicates that the motion vector of the previous field p1 has been searched, and the motion vector of the first-cycle p1 field is generally initialized to be all 0 vectors) as a time domain candidate vector of the current field, taking a segmented image block as a basic unit and a p2 field image signal as a reference field, and waiting for selection and optimization of multiple types of candidate vectors based on a spatial domain, a time domain, 0Mv, and a random motion selection vector to perform homopolar field 3DRS motion estimation, where p1 represents a previous field image, p2 represents a previous field image, and p2' represents a previous frame image obtained after motion compensation.
Fig. 4 is a flowchart of the same polarity field 3DRS motion estimation in step S102 according to an embodiment of the present invention. As shown in fig. 4, step S102 further includes:
step S2.1, multi-class candidate vector selection, namely according to the current image block cfblkThe multi-class candidate vector selection of the spatial domain candidate vector, the temporal domain candidate vector, the 0Mv candidate vector and the random motion candidate vector is performed, and a detailed schematic diagram of the multi-class candidate vector selection is shown in fig. 5. In particular, the amount of the solvent to be used,
s0, S1, S2 and S3 respectively represent motion vectors of blocks, called current blocks cf, of which the current fields of the current block, i.e. the m-1 st column, the m +1 st column and the m-1 st column in the n-1 st row, have been searched out from the fields with the same polarityblkThe spatial candidate vector of (1).
T0, T1, T2, T3, T4, T5, T6, T7 and T8 are respectively expressed as block motion vectors searched by the motion estimation of the heteropolar field of the previous field p1 of the (n-1) th line, the (n-1) th column, the (n-1) th line, the (m + 1) th column, the (n + 1) th line, the (m-1) th column, the (n + 1) th line and the (m + 1) th column of the current block, which are called current blocks cfblkThe time domain candidate vector of (2).
0Mv denotes a motion vector with a motion size of 0, i.e. 0Mv is (0,0), which is referred to as the current block cfblk0Mv candidate vector.
Rnd0 and Rnd1 are respectively expressed as random motion vectors, and the magnitude sign of the random motion vectors is denoted as RandMv ═ (x, y), where x belongs to the motion vector horizontal direction search window [ -32,32]Random value in the range, y belongs to the motion vector vertical search window [ -16,16 [ ]]Random values in the range named Rnd0 and Rnd1 as the current block cfblkThe random motion candidate vector of (2).
Step S2.2, the SAD of each candidate vector is calculated. In particular, according to the current image block cfblkAnd the motion of spatial, temporal, 0Mv, and random motion candidate vectorsThe dynamic sizes are respectively projected on a field p2 with the same polarity to obtain a corresponding image block p2blkRespectively calculating its corresponding image block and its preceding image block cfblkThe symbol is denoted as SAD, and the calculation formula is as follows:
according to the SAD value, if the SAD value is smaller, the time domain distance between the corresponding reference image block and the current image block is more similar, namely the matching degree between the reference image block and the current image block is higher; if the SAD value is larger, it indicates that the temporal distance between the corresponding reference image block and the current image block is different, that is, the reference image block and the current image block are matched to a lower degree.
According to the formula (1), the SAD value of each candidate vector can be calculated, and the sign is respectively marked as SADS0、...、 SADS3、SADT0、SADT1、...、SADT8、SAD0Mv、SADRnd0、SADRnd1。
And step S2.3, calculating the penalty of each candidate vector according to the SAD of each candidate vector.
Theoretically, the matching degree of the real motion vector of the current image block corresponding to the reference image block is the highest, that is, the corresponding SAD value is the smallest, but since the image is affected by various links of the whole video channel, such as atmospheric noise at the video shooting end, noise of a shooting system, video compression and decompression in video transmission, communication transmission of the video and the like, the change of the image pixel value is affected and changed, sampling errors and coding and decoding errors are easy to occur at the high frequency part of the image, noise interference is easy to occur at the low frequency part of the image, and a fragment effect caused by the division of the image block, the corresponding SAD value of the real motion vector is not necessarily the smallest. If the motion vector field is directly searched only according to the size of the SAD value, the consistency of the motion vector field is very poor, and particularly, the multiple and disordered characteristics of the motion vector can occur at the boundary of an object, so that errors such as flicker, jitter, feathering and the like of the image effect of motion compensation can be caused. Therefore, the SAD value needs to be constrained according to the morphological characteristics of local continuity and smooth fragmentation of the motion vector, namely, the penalty is increased, so that the motion vector searched by the 3DRS has good consistency in a local space, the motion consistency of a de-interlaced image is increased, and the phenomena of image flicker and image jitter are reduced.
In the embodiment of the present invention, the penalty calculation process is as follows:
firstly, respectively calculating respective penalty factors according to the types of the candidate vectors, wherein the formula is as follows:
in which ηSFor pre-set threshold, lambda, of spatial domain candidate vectorsSiFor a predetermined coefficient factor, alpha, of the spatial candidate vectorSA penalty factor which is a space domain candidate vector; etaTFor a predetermined threshold value, λ, of the time-domain candidate vectorTiFor predetermined coefficient factors, alpha, of time-domain candidate vectorsTA penalty factor being a time domain candidate vector; eta0Preset threshold value, lambda, for a 0Mv candidate vector0Predetermined coefficient factor, alpha, for a 0Mv candidate vector0A penalty factor of a 0Mv candidate vector; etaRA predetermined threshold value, λ, for a random motion candidate vectorRiPredetermined coefficient factor, alpha, for random motion candidate vectorsRA penalty factor for a random motion candidate vector.
Next, the distance between each candidate vector and the neighborhood motion vector of the current block is calculated, as shown in fig. 6, the calculation formula is as follows:
D＝||Mv-S0||+||Mv-S2||+||Mv-T6||+||Mv-S8|| (3)
where Mv denotes a candidate vector, S0, S2, T6, and T8 denote neighborhood motion vectors of the current image block, and D denotes a distance between the Mv candidate vector and the neighborhood motion vector of the current block.
According to the formula (3), the distance between each candidate vector and the neighboring motion vector can be calculated, and the sign is respectively recorded as: dS0、...、DS2、DT0、...、DT8、D0Mv、DRnd0、DRnd1。
And finally, according to the punishment quantity of each candidate vector, the calculation formula is as follows:
P＝α·D (4)
where α represents a penalty factor for the candidate motion vector, D represents a neighborhood vector distance for the candidate motion vector, and P represents a penalty amount for the candidate vector, such as: pS0＝αS0·DS0And the other candidate vectors are similar.
According to the formula (4), the penalty of each candidate vector can be calculated, and the sign is respectively recorded as: pS0、...、PS2、PT0、...、PT8、P0Mv、PRnd0、PRnd1。
And S2.4, selecting the optimal candidate vector according to the SAD value of each candidate vector and the penalty of each candidate vector.
Specifically, step S2.4 processes as follows:
firstly, according to the calculation results of step S2.2 and step S2.3, the corresponding image block matching value of each candidate vector can be obtained, and the calculation formula is as follows:
SADP＝SAD+P (5)
in the equation, SAD represents the SAD value of each candidate vector calculated in step S2.2, and P represents the penalty of each candidate vector calculated in step S2.3, for example:
other candidate vectors are similar.
According to the formula (5), SAD after each candidate vector constraint can be calculatedPThe values, symbols are respectively noted as:
next, the constrained SAD is determined according to each candidate vectorPValue, selecting the best candidate vectorThe calculation formula is as follows:
SADminthe corresponding candidate vector is the optimal candidate vector selected finally, and the symbol is marked as MvCand。
And S2.5, calculating an updating step-size factor, namely a self-adaptive searching step-size factor according to the selected optimal candidate vector.
If there is no true motion vector of the current image block in the multi-class candidate vectors, if only 2 random candidate vectors are used for iteration and hit, the convergence rate of the 3DRS is very slow and will not converge in special cases, especially for short-term large motion of an object, such as a fighting motion segment in a motion slice. Therefore, a mechanism is needed to be added to ensure that the convergence speed of the 3DRS search is accelerated, the invention provides a dynamic self-adaptive variable step length-based Update (UPDATE) strategy of the 3DRS, and the 3DRS can be ensured to be converged quickly. The core idea is to judge the current last candidate vector MvCandSAD ofminValue size if SADminThe smaller value indicates MvCandThe corresponding reference image block has high matching degree with the current block, and the real motion vector is the same as MvCandThe difference is small, only MvCandThe real motion vector can be easily obtained by searching nearby once again; if SADminThe value is relatively large, which indicates MvCandThe matching degree of the corresponding reference image block and the current block is low, and the real motion vector is the same as MvCandThe difference is large, at MvCandThere is no real motion vector in the vicinity, and it is required at MvCandThe real motion vector can be obtained only by searching for a place far away; so according to SADminAnd dynamically and adaptively adjusting and updating the search step size of the value to search the motion vector again. The Update search step factor calculation formula is as follows:
β＝max[1,min(βmax,λ·SADmin)] (7)
wherein λ is a predetermined coefficient of the step factor, βmaxBeta represents the adaptive search step-size factor, which is a preset maximum threshold value of the step-size factor.
And S2.6, selecting an optimal updating candidate vector according to the updating candidate base vector and the self-adaptive searching step-size factor, and outputting the optimal updating candidate vector to a block motion vector cache of the current field to be used as a space domain candidate vector and a neighborhood candidate vector of a next image block and a time domain candidate vector of a next image field.
First, referring to fig. 7, Update candidate base vectors are (-1, -1), (-1,0), (-1,1), (0, -1), (0,1), (1, -1), (1,0), (1,1), and symbols are denoted as (1, -1), (1,1), respectively
From the Update candidate base vector and the adaptive search step factor, an Update candidate vector can be calculated as follows:
in the formula, the raw materials are mixed,
expressed as the ith Update candidate base vector, UiRepresents the Update candidate vector after the adaptive step-size factor adjustment.
Next, Update candidate vectors U are calculated according to the calculation procedure of step S2.2iThe corresponding SAD values, signs are respectively noted
Where i ∈ {0, 1.., 6 }.
Finally, an optimal Update candidate vector is selected according to the SAD value of the Update candidate vector, and the calculation formula is as follows:
SADbestthe corresponding vector is the optimal Update candidate vector selected finally, and the symbol is marked as MvbestAnd the motion vector is output to a block motion vector cache of the current field to be used as a spatial domain candidate vector and a neighborhood candidate vector of a next image block and a temporal domain candidate vector of a next image field.
In step S103, the same-polarity Block Motion Vector (Block Motion Vector) field of the current field cf is output. The invention carries out block matching by taking the block level as a unit, can reduce the negative influence of noise on motion estimation, increases the motion vector search stability of 3DRS, can realize dimension reduction on a vector field of an image, and can greatly reduce the consumption of hardware resources (control hardware cost).
Step S2, perform a second search for the 3DRS motion estimation between the images p2 'and p1, and further perform a precise motion vector search to converge it to the true motion vector, so as to obtain a motion vector field (i.e., a field motion vector field with opposite polarity) of the current interpolated field p1, where p1 represents the currently input field image and p2' represents the previous frame image obtained after motion compensation.
It should be noted here that, in the present invention, the "current field" is for the current processing procedure, since the interpolation compensation and the motion estimation are performed on different field images, the current processing procedure in this step is a heteropolar field motion vector search, that is, a heteropolar field motion vector search is performed between p2' and p1, and a motion vector on a p1 field image is solved, so that p1 is called the heteropolar motion estimated current field image, and for the following motion compensation, the p1 field image is also subjected to motion compensation processing, so that p1 is called the motion compensated current field image, that is, the current interpolated field image. In the motion estimation in step S1, the cf field image is referred to as the current field image of the motion estimation, and the motion vector of the cf field image is searched for by referring to the p2 field image (cf and p2 are homopolar relationships, p1 and p3 are homopolar relationships, cf and p1, p1 and p2, and p2 and p3 are heteropolar relationships, in short, the interpolar relationships are the homopolar relationships between the interlaced fields, and the heteropolar relationships between the adjacent fields), and in the motion compensation, the p1 field image is the current field image of the motion compensation, and the motion vector of the p1 field image is used, and the motion compensation process is performed by referring to the p 2'.
Specifically, as shown in fig. 8, step S2 further includes the steps of:
in step S201, the image blocks (H ') are mapped to the input field image p1'blk×W′blkA block of pixels of a size) unit segmentation. In this embodiment, p1 represents a previous field image, that is, a currently input field image, p2 represents a previous field image, and p2 'represents a previous frame image obtained after motion compensation, that is, a reference frame image for motion estimation of an opposite-polarity field (a result of de-interlacing of the previous field, where the 1 st field is an intra-field interpolation de-interlacing result substitute), in the embodiment of the present invention, the resolution of the p1 field image is 1920 × 540 pixels, and the resolution of the p2' frame image is 1920 × 1080 pixels, and the image is unit-segmented according to the foregoing step 101, which is not described herein again.
Step S202, performing intra-field interpolation on the field-level image blocks to obtain frame-level image blocks (i.e. 2H'blk×W′blkA block of pixels of size); specifically, an image block in a p1 field image is taken, the resolution is 8 × 4 pixels, two-dimensional image interpolation is performed in the vertical direction of the image block, a specific method can select bilinear interpolation, bicubic interpolation, directional interpolation and the like, and the resolution of the image block after interpolation is 8 × 8 pixels.
In step S203, heteropolar field 3DRS Motion estimation is performed with the image blocks after intra-field interpolation as the basic units, p2' as the reference field, and p1 field Block Motion vectors (Block Motion Vector) as the time domain candidate vectors. Specifically, the current block image is an 8 × 8 pixel block interpolated in the vertical direction of the image block in the p1 field image, p2' is a reference frame image, and the p1 field block motion vector field is a time domain candidate vector, and a homopolar field motion vector field is calculated according to the similar process of the foregoing step S1, so as to implement heteropolar field 3DRS motion estimation, which is not described herein again.
In step S204, the Block Motion Vector field (Block Motion Vector) of the opposite polarity field p1 is output. Therefore, the heteropolar field 3DRS motion estimation of the invention carries out heteropolar field motion estimation on p1 and p2' on the basis of homopolar field motion estimation, and simultaneously searches again based on the motion vector of the heteropolar field, so that the motion vector result can be further accurate, and the problems of de-interlacing and flickering of a repeated frame scene of the homopolar field and the problems of flickering, error, sawtooth and the like of de-interlacing caused by the inconsistency of the motion vectors of the homopolar field and the heteropolar field in the time domain are solved.
Step S3, projecting the motion vector of the field with opposite polarity onto the reference frame p2', finding the best matched reference block pixel with the current block to perform motion compensated de-interlacing on the current compensated interpolated field p1, and outputting the current de-interlacing result, which is denoted as p 1'.
Specifically, according to the accurate motion vector searched by the motion estimation based on the field with different polarity, the motion compensation de-interlacing is performed on the p1 field image, as shown in fig. 9, the current block motion vector Mv ═ x, y is projected onto the reference frame p2', the pixel value projected on p2' is found and compensated to the position of the corresponding interlacing of p 1.
Fig. 10 is a system architecture diagram of a motion compensation de-interlacing system based on motion estimation of bipolar field according to the present invention, and fig. 11 is a schematic diagram of the motion compensation de-interlacing system based on motion estimation of bipolar field in fig. 10. As shown in fig. 10 and 11, a motion compensated de-interlacing system for motion estimation based on bipolar fields according to the present invention includes:
a homopolar field motion estimation unit 90, configured to perform a first search on 3DRS motion estimation between input homopolar field images cf and p2 to obtain a converged current field cf motion vector field (homopolar field motion vector field).
Specifically, as shown in fig. 12, the homopolar field motion estimation unit 90 includes:
a first field image blocking unit 901 for blocking the input field image cf in image blocks (H'blk×W′blkA block of pixels of a size) unit segmentation; cf (current field) represents the current field picture.
A homopolar field 3DRS Motion estimation unit 902, configured to perform homopolar field 3DRS Motion estimation on an input p1 field Block Motion Vector (Block Motion Vector) as a temporal candidate Vector of a current field, with each sliced image Block as a basic unit and p2 as a reference field, where p1 denotes a previous field image, p2 denotes a previous field image, and p2' denotes a previous frame image obtained after Motion compensation. The homopolar field 3DRS motion estimation unit 902 further includes:
a multi-class candidate vector selecting unit 902a for selecting a candidate vector according to the current image block cfblkSelecting a plurality of types of candidate vectors of a space domain candidate vector, a time domain candidate vector, a 0Mv candidate vector and a random motion candidate vector;
a SAD calculation unit 902b for calculating SAD of each candidate vector. Specifically, the SAD calculation unit 902b calculates the current image block cf based on the current image block cfblkAnd the motion magnitudes of the spatial domain candidate vector, the temporal domain candidate vector, the 0Mv candidate vector and the random motion candidate vector are respectively projected onto a homopolar field p2 to obtain a corresponding image block p2blkRespectively calculating its corresponding image block and current image block cfblkThe symbol is denoted as SAD, and the calculation formula is as follows:
according to the SAD value, if the SAD value is smaller, the time domain distance between the corresponding reference image block and the current image block is more similar, namely the matching degree between the reference image block and the current image block is higher; if the SAD value is larger, it indicates that the temporal distance between the corresponding reference image block and the current image block is different, that is, the reference image block and the current image block are matched to a lower degree.
Calculating SAD value of each candidate vector according to formula, and marking symbols as SADS0、...、 SADS3、SADT0、SADT1、...、SADT8、SAD0Mv、SADRnd0、SADRnd1。
A penalty calculation unit 902c for calculating a penalty P of each candidate vector according to the SAD value of each candidate vector;
firstly, calculating respective penalty factors according to the types of the candidate vectors
Secondly, calculating the distance between each candidate vector and the neighborhood motion vector of the current block;
and finally, respectively calculating the penalty of each candidate vector according to the penalty factor of each candidate motion vector and the neighborhood vector distance of each candidate motion vector.
An optimal candidate vector selecting unit 902d is configured to select an optimal candidate vector according to the SAD of each candidate vector and the penalty P of each candidate vector.
Specifically, the optimal candidate vector selection unit 902d performs the following process:
first, the corresponding image block matching value SAD of each candidate vector can be obtained from the calculation results of the SAD calculation unit 902b and the penalty calculation unit 902cP；
Next, the constrained SAD is determined according to each candidate vectorPValue, selecting the optimal candidate vector SADminSymbol MvCand。
And an update compensation factor calculation unit 902e, configured to calculate an adaptive search step factor β according to the optimal candidate vector.
If there is no true motion vector of the current image block in the multi-class candidate vectors, if only 2 random candidate vectors are used for iteration and hit, the convergence rate of the 3DRS is very slow and will not converge in special cases, especially for short-term large motion of an object, such as a fighting motion segment in a motion slice. Therefore, a mechanism is needed to be added to ensure that the convergence speed of the 3DRS search is accelerated, the invention provides a dynamic self-adaptive variable step length-based Update (UPDATE) strategy of the 3DRS, and the 3DRS can be ensured to be converged quickly. The core idea is to judge the current last candidate vector MvCandSAD ofminValue size if SADminThe smaller value indicates MvCandThe corresponding reference image block has high matching degree with the current block, and the real motion vector is the same as MvCandThe difference is small, only needTo be at MvCandThe real motion vector can be easily obtained by searching nearby once again; if SADminThe value is relatively large, which indicates MvCandThe matching degree of the corresponding reference image block and the current block is low, and the real motion vector is the same as MvCandThe difference is large, at MvCandThere is no real motion vector in the vicinity, and it is required at MvCandThe real motion vector can be obtained only by searching for a place far away; so according to SADminAnd dynamically and adaptively adjusting the Update search step size according to the value to search the motion vector again.
And an optimal update vector selection unit 902f, configured to select an optimal update candidate vector according to the update candidate base vector and the adaptive search step factor β. The process of the optimal update vector selection unit 902f is specifically as follows:
first, an Update candidate vector U can be calculated from the Update candidate base vector and the adaptive search step factori；
Next, Update candidate vectors U are calculated respectively according to the calculation process of the SAD calculation unit 902biThe corresponding SAD values, signs are respectively noted
Where i ∈ {0, 1.., 6 }.
Finally, the optimum Update candidate vector SAD is selected according to the SAD value of the Update candidate vectorbestSymbol MvbestAnd the motion vector is output to a block motion vector cache of the current field to be used as a spatial domain candidate vector and a neighborhood candidate vector of a next image block and a temporal domain candidate vector of a next image field.
A first output unit 903 for outputting a block motion vector field of the current field cf. The invention carries out block matching by taking the block level as a unit, can reduce the negative influence of noise on motion estimation, increases the motion vector search stability of 3DRS, can realize dimension reduction on a vector field of an image, and can greatly reduce the consumption of hardware resources (control hardware cost).
The heteropolar field motion estimation unit 91 is configured to perform a second search on the 3DRS motion estimation between the images p2' and p1, and further perform a precise motion vector search to converge the motion vector to a true motion vector, so as to obtain a motion vector field (heteropolar field motion vector field) of the current interpolated field p 1. Specifically, as shown in fig. 13, the heteropolar field motion estimation unit 91 further includes:
a second image block segmentation unit 910 for segmenting the input field image p1 into image blocks (H'blk×W′blkA block of pixels of a size) unit segmentation; specifically, p1 is the currently input field image, p2 'is the reference frame image of the heteropolar field motion estimation (the result of the de-interlacing of the previous field, and the 1 st field is replaced by the de-interlacing result of the intra-field interpolation), in this embodiment of the present invention, the resolution of the p1 field image is 1920 × 540 pixels, and the resolution of the p2' frame image is 1920 × 1080 pixels, and the image is unit-segmented according to the foregoing step 101, which is not described herein again.
A field image block interpolation unit 911 for performing intra-field interpolation on the field-level image blocks to obtain frame-level image blocks (2H'blk×W′blkA block of pixels of size); specifically, an image block in a p1 field image is taken, the resolution is 8 × 4 pixels, two-dimensional image interpolation is performed in the vertical direction of the image block, a specific method can select bilinear interpolation, bicubic interpolation, directional interpolation and the like, and the resolution of the image block after interpolation is 8 × 8 pixels.
And a heteropolar field 3DRS Motion estimation unit 912, configured to perform heteropolar field 3DRS Motion estimation by using the image Block after intra-field interpolation as a basic unit, p2' as a reference field, and p1 field Block Motion vectors (Block Motion vectors) as time domain candidate vectors. Specifically, the current block image is an 8 × 8 pixel block interpolated in the vertical direction of the image block in the p1 field image, p2' is a reference frame image, and the p1 field block motion vector field is a time domain candidate vector, and a homopolar field motion vector field is calculated according to the similar process of the homopolar field 3DRS motion estimation unit 902, so as to implement heteropolar field 3DRS motion estimation, which is not described herein again.
A second output unit 913 for outputting the heteropolar block motion vector field of the current field p 1. According to the motion estimation of the 3DRS in the different-polarity field, on the basis of the motion estimation of the same-polarity field, the motion estimation of the different-polarity field is carried out on p1 and p2', meanwhile, the motion vector is searched again on the basis of the motion vector of the different-polarity field, the motion vector result can be further accurate, and the problems of de-interlacing flicker of repeated frame scenes in the same-polarity field and de-interlacing flicker, errors, sawteeth and the like caused by the fact that the motion vectors of the same-polarity field and the motion vectors of the different-polarity field are not consistent in the time domain are solved.
And a motion compensation de-interlacing unit 92, configured to project the motion vector of the field with opposite polarity onto the reference frame p2', find a best matching reference block pixel with the current block, perform motion compensation de-interlacing on the current compensated interpolated field p1, and output a current de-interlacing result, which is denoted as p 1'.
Specifically, the motion compensation de-interlacing unit 92 performs motion compensation de-interlacing on the p1 field image according to the accurate motion vector searched out based on the motion estimation of the opposite polarity field, projects the current block motion vector Mv ═ x, y onto the reference frame p2', finds the pixel value projected on p2' and compensates to the corresponding interlacing position of p 1. The accurate motion vector searched based on the motion estimation of the heteropolar field is used for carrying out motion compensation de-interlacing, so that the problems of flicker, error, sawtooth and the like caused by the fact that the motion vectors of the homopolar field and the heteropolar field are different in the time domain are well solved.
It should be noted that, in fig. 11, display indicates that p1 'is obtained after motion compensation deinterlacing is performed on p1, and then p1' is sent to a display for display, and Online indicates that the Online input of cf field images at the video acquisition end can be supported, which means that hardware resources are saved compared with a mode of inputting cf field images from a memory, and a structure for realizing motion estimation of the bipolar field by hardware is also facilitated, and is not described herein again.
In summary, the motion compensation de-interlacing method and system based on motion estimation of a bipolar field of the present invention perform a first motion vector search through motion estimation of a 3DRS of a same polarity field, so as to rapidly converge a motion vector field, then perform a second motion vector search through motion of a 3DRS of a different polarity field, further refine a motion vector of motion compensation of a current field, so as to converge the motion vector to a real motion vector, and finally perform motion compensation interpolation de-interlacing processing according to a vector field searched by the 3DRS of the different polarity field, thereby solving the de-interlacing flicker problem phenomenon of a repeated frame scene of the same polarity field, and the de-interlacing problem phenomena of errors, sawteeth and feathering caused by inconsistency of the motion vector of the same polarity field and the motion vector of the different polarity field in a time domain.
Compared with the prior art, the invention has the following advantages:
1. according to the correlation of a time domain and a space domain of a motion vector field, the invention provides the selection and optimization of a plurality of types of motion candidate vectors based on space domain, time domain, 0Mv and random motion candidate vectors, greatly accelerates the convergence speed of the 3DRS, is one of key points of the invention, and solves the problem of difficulty in slow convergence speed of the previous method for searching the motion vectors based on the 3 DRS.
2. The invention provides morphological characteristic constraint based on local continuity of motion vectors and smooth fragmentation, so that the motion vectors searched by the 3DRS have good consistency in a local space, the motion consistency of a de-interlaced image is increased, and the phenomena of image flicker and jitter are reduced.
3. The invention provides a dynamic self-adaptive variable step length-based 3DRS updating strategy, which can further greatly accelerate the convergence speed of the 3 DRS.
4. On the basis of the motion vector result of 3DRS search, the invention simultaneously searches again based on the motion vector of the heteropolar field, can further accurately obtain the motion vector result, and solves the problems of flicker, error, sawtooth and the like caused by the fact that the motion vectors of the heteropolar field are inconsistent in the time domain and the like in the de-interlacing due to the repeated frame scene of the homopolar field.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Modifications and variations can be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the present invention. Therefore, the scope of the invention should be determined from the following claims.
Claims (9)
1. A motion compensated de-interlacing method based on motion estimation of bipolar fields, comprising the steps of:
step one, performing 3DRS motion estimation between input homopolar field images cf and p2, and performing a first search to obtain a homopolar field motion vector field of a converged current field cf, where cf represents a current field image, and p2 represents a reference field image for motion estimation:
segmenting an input field image cf according to an image block unit, wherein cf is a current field image;
taking an input p1 field block motion vector as a time domain candidate vector of a current field, taking a segmented image block as a basic unit and a p2 field image signal as a reference field, and selecting and optimizing multi-class candidate vectors based on a space domain, a time domain, 0Mv and a random motion candidate vector to carry out homopolar field 3DRS motion estimation, wherein p1 represents a previous field image, and p2 represents a previous field image;
outputting the same-polarity block motion vector field of the current field cf;
secondly, performing second search on 3DRS motion estimation between the images p2 'and p1, further performing accurate motion vector search to make the images converge on a real motion vector, and obtaining a heteropolar field motion vector field of the current interpolation field p1, wherein p1 represents a currently input field image, and p2' is a reference frame image of the heteropolar field motion estimation;
and step three, projecting the motion vector of the field with different polarity onto a reference frame p2', finding a reference block pixel which is best matched with the current block to perform motion compensation de-interlacing on the current compensation interpolation field p1, and outputting a current de-interlacing result.
2. The method for motion compensated de-interlacing based on motion estimation of bi-polar fields according to claim 1, characterized in that the homopolar field 3DRS motion estimation further comprises the steps of:
step S2.1, according to the segmented current image block cfblkSelecting a plurality of types of candidate vectors of a space domain candidate vector, a time domain candidate vector, a 0Mv candidate vector and a random motion candidate vector;
step S2.2, calculating SAD value of each candidate vector;
s2.3, calculating the punishment quantity of each candidate vector according to the SAD value of each candidate vector;
s2.4, selecting an optimal candidate vector according to the SAD value of each candidate vector and the penalty of each candidate vector;
s2.5, calculating a self-adaptive search step size factor according to the selected optimal candidate vector;
and S2.6, selecting an optimal updating candidate vector according to the updating candidate base vector and the self-adaptive searching step-size factor, and outputting the optimal updating candidate vector to a block motion vector cache of the current field to be used as a space domain candidate vector, a neighborhood candidate vector and a time domain candidate vector of the next image block.
3. The method for motion compensated de-interlacing based on motion estimation of bi-polar fields as claimed in claim 2, characterized in that in step S2.2 the current image blocks cf are based onblkAnd the motion magnitudes of the spatial domain candidate vector, the temporal domain candidate vector, the 0Mv candidate vector and the random motion candidate vector are respectively projected onto a homopolar field p2 to obtain a corresponding image block p2blkRespectively calculating its corresponding image block and its preceding image block cfblkThe symbol is denoted as SAD.
4. The method for motion compensated de-interlacing based on motion estimation of bi-polar fields as claimed in claim 2, characterized in that step 2.3 further comprises:
respectively calculating respective penalty factors according to the types of the candidate vectors;
calculating the distance between each candidate vector and the neighborhood motion vector of the current block;
and respectively calculating the penalty amount of each candidate vector according to each penalty factor and the distance between each candidate vector and the neighborhood motion vector of the current block.
5. Motion compensated de-interlacing method for motion estimation based on bipolar fields as claimed in claim 1, characterized in that step S2.4 further comprises:
based on the SAD of each candidate vector obtained by calculationAnd the punishment quantity of each candidate vector, and calculating to obtain the image block matching value SAD corresponding to each candidate vectorP；
According to the constrained SAD of each candidate vectorPAnd selecting the optimal candidate vector.
6. The method for motion compensated de-interlacing based on motion estimation of bi-polar fields as claimed in claim 1, characterized in that in step S2.5, the update search step factor calculation formula is as follows:
β＝max[1,min(βmax,λ·SADmin)]
wherein, λ is a preset coefficient of the step factor, βmaxFor a preset maximum threshold value of the step-size factor, beta denotes the adaptive search step-size factor, SADminIs the optimal candidate vector.
7. The method for motion compensated de-interlacing based on motion estimation of bi-polar fields as claimed in claim 1, characterized in that step 2.6 further comprises:
calculating an updated candidate vector U according to the updated candidate base vector and the self-adaptive search step size factori；
Respectively calculating an update candidate vector U according to the step S2.2iThe corresponding SAD value;
and selecting the optimal updating candidate vector according to the SAD value of the updating candidate vector.
8. The motion compensated de-interlacing method for motion estimation based on bi-polar fields as claimed in claim 1, wherein the step two further comprises the steps of:
splitting an input field image p1 in units of image blocks, wherein p1 represents a previous field image;
performing field interpolation on the field-level image blocks to obtain frame-level image blocks;
taking the image block after the intra-field interpolation as a basic unit, p2' as a reference field, and taking the input p1 field block motion vector as a time domain candidate vector, and performing heteropolar field 3DRS motion estimation; wherein p2' is a reference frame image for heteropolar field motion estimation;
the block motion vector field of the opposite polarity field p1 is output.
9. A motion compensated de-interlacing system based on motion estimation of bipolar fields, comprising:
a homopolar field motion estimation unit, configured to perform a first search on 3DRS motion estimation between input homopolar field images cf and p2 to obtain a homopolar field motion vector field of a converged current field cf, where cf represents a current field image, and p2 represents a reference field image for motion estimation:
segmenting an input field image cf according to an image block unit, wherein cf is a current field image;
taking an input p1 field block motion vector as a time domain candidate vector of a current field, taking a segmented image block as a basic unit and a p2 field image signal as a reference field, and selecting and optimizing multi-class candidate vectors based on a space domain, a time domain, 0Mv and a random motion candidate vector to carry out homopolar field 3DRS motion estimation, wherein p1 represents a previous field image, and p2 represents a previous field image;
outputting the same-polarity block motion vector field of the current field cf;
the heteropolar field motion estimation unit is used for carrying out second search on 3DRS motion estimation between the images p2 'and p1, further accurately searching a motion vector to make the motion vector converge on a real motion vector, and obtaining a heteropolar field motion vector field of the current interpolation field p1, wherein p1 represents a currently input field image, and p2' represents a reference frame image of the heteropolar field motion estimation;
and the motion compensation de-interlacing unit is used for projecting the motion vectors of the fields with different polarities onto the reference frame p2', finding out the reference block pixel which is best matched with the current block to perform motion compensation de-interlacing on the current compensation interpolation field p1, and outputting a current de-interlacing result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810117131.6A CN108282653B (en) | 2018-02-06 | 2018-02-06 | Motion compensation de-interlacing method and system based on motion estimation of bipolar field |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810117131.6A CN108282653B (en) | 2018-02-06 | 2018-02-06 | Motion compensation de-interlacing method and system based on motion estimation of bipolar field |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108282653A CN108282653A (en) | 2018-07-13 |
| CN108282653B true CN108282653B (en) | 2021-09-03 |
Family
ID=62807677
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810117131.6A Active CN108282653B (en) | 2018-02-06 | 2018-02-06 | Motion compensation de-interlacing method and system based on motion estimation of bipolar field |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108282653B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109756778B (en) * | 2018-12-06 | 2021-09-14 | 中国人民解放军陆军工程大学 | Frame rate conversion method based on self-adaptive motion compensation |
| CN109951713B (en) * | 2019-03-11 | 2023-02-28 | 深圳信息职业技术学院 | Motion estimation compensation circuit and method for video de-interlacing |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101510985A (en) * | 2009-02-27 | 2009-08-19 | 西安交通大学 | Self-adapting de-interleave method for movement compensation accessory movement |
| CN102364933A (en) * | 2011-10-25 | 2012-02-29 | 浙江大学 | Motion-classification-based adaptive de-interlacing method |
| US8503533B1 (en) * | 2008-02-01 | 2013-08-06 | Zenverge, Inc. | Motion estimation engine for performing multiple types of operations |
| CN106303340A (en) * | 2016-09-20 | 2017-01-04 | 天津大学 | A kind of bi-directional motion estimation interlace-removing method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100309372A1 (en) * | 2009-06-08 | 2010-12-09 | Sheng Zhong | Method And System For Motion Compensated Video De-Interlacing |
-
2018
- 2018-02-06 CN CN201810117131.6A patent/CN108282653B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8503533B1 (en) * | 2008-02-01 | 2013-08-06 | Zenverge, Inc. | Motion estimation engine for performing multiple types of operations |
| CN101510985A (en) * | 2009-02-27 | 2009-08-19 | 西安交通大学 | Self-adapting de-interleave method for movement compensation accessory movement |
| CN102364933A (en) * | 2011-10-25 | 2012-02-29 | 浙江大学 | Motion-classification-based adaptive de-interlacing method |
| CN106303340A (en) * | 2016-09-20 | 2017-01-04 | 天津大学 | A kind of bi-directional motion estimation interlace-removing method |
Non-Patent Citations (2)
| Title |
|---|
| De-interlacing algorithms based on motion compensation;Xinbo Gao等;《 IEEE Transactions on Consumer Electronics》;20050718;全文 * |
| 一种改进的三维递归搜索视频去隔行算法;徐洪峰;《计算机应用》;20070531;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108282653A (en) | 2018-07-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR100530223B1 (en) | Frame interpolation method and apparatus at frame rate conversion | |
| JP4159606B2 (en) | Motion estimation | |
| US6192079B1 (en) | Method and apparatus for increasing video frame rate | |
| US8718143B2 (en) | Optical flow based motion vector estimation systems and methods | |
| EP1638339B1 (en) | Motion estimation | |
| US8514939B2 (en) | Method and system for motion compensated picture rate up-conversion of digital video using picture boundary processing | |
| KR20100114499A (en) | Image interpolation with halo reduction | |
| US20050249288A1 (en) | Adaptive-weighted motion estimation method and frame rate converting apparatus employing the method | |
| CN103826133A (en) | Method and apparatus for motion compensated frame rate up conversion | |
| CN108574844B (en) | Multi-strategy video frame rate improving method for space-time significant perception | |
| KR20090041562A (en) | Frame interpolating device and frame rate up-converting apparatus having the same | |
| CN104811726B (en) | The candidate motion vector selection method of estimation in frame per second conversion | |
| CN108282653B (en) | Motion compensation de-interlacing method and system based on motion estimation of bipolar field | |
| KR20050080862A (en) | Method for interpolating frame with motion compensation by overlapped block motion estimation and frame-rate converter using thereof | |
| JP2005318622A (en) | Reverse film mode extrapolation method | |
| KR100968642B1 (en) | Method and interpolation device for calculating a motion vector, display device comprising the interpolation device, and computer program | |
| JP2005318623A (en) | Film mode extrapolation method, film mode detector, and motion compensation apparatus | |
| US8305490B2 (en) | De-interlacing system | |
| Jing et al. | Smooth constrained motion estimation for video coding | |
| KR100587263B1 (en) | Method for frame rate conversing of video signal | |
| Zhu et al. | Fast layered bit-plane matching for electronic video stabilization | |
| CN110381324A (en) | A kind of frame interpolated value interlace-removing method based on edge detection | |
| EP2400740A2 (en) | Method and device for estimating motion vectors for frame interpolation | |
| Kang et al. | Hardware implementation of motion estimation using a sub-sampled block for frame rate up-conversion | |
| Ghutke | Temporal video frame interpolation using new cubic motion compensation technique |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |